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Preface 


This book started from thè premise that Computer Science should be taught as 
a liberal art, not an industriai skill. I had thè privilege of taking 6.001 from Gerry 
Sussman when I was a first year student at MIT, and that course awakened me 
to thè power and beauty of computing, and inspired me to pursue a career as 
a teacher and researcher in Computer Science. When I arrived as a new faculty 
member at thè University of Virginia in 1999, I was distraught to discover that 
thè introductory computing courses focused on teaching industriai skills, and 
with so much of thè course time devoted to explaining thè technical complexi- 
ties of using bloated industriai languages like C++ and Java, there was very little, 
if any, time left to get across thè core intellectual ideas that are thè essence of 
computing and thè reason everyone should learn it. 

With thè help of a University Teaching Fellowship and National Science Foun- 
dation grants, I developed a new introductory computer Science course, tar- 
geted especially to students in thè College of Arts & Sciences. This course was 
first offered in Spring 2002, with thè help of an extraordinary group of Assistant 
Coaches. Because of some unreasonable assumptions in thè first assignment, 
half thè students quickly dropped thè course, but a small, intrepid, group of pi- 
oneering students persisted, and it is thanks to their efforts that this book exists. 
That course, and thè next several offerings, used Abelson & Sussman’s outstand- 
ing Structure and Interpretation of Computer Programs (SICP) textbook along 
with Douglas Hofstadter’s Godei, Escher, Bach: An Eternai Golden Braid. 



Spring 2002 CS200 Pioneer Graduates 

Back row, from left: Portman Wills ( Assistant Coach), Spencer Stockdale, Shawn O’Hargan, 

Jeff Taylor, Jacques Fournier, Katie Winstanley, Russell O’Reagan, Victor Clay Yount. 

Front: Grace Deng, Rachel Dada, Jon Erdman ( Assistant Coach). 

I am not alone in thinking SICP is perhaps thè greatest textbook ever written in 
any fìeld, so it was with much trepidation that I endeavored to develop a new 
textbook. I hope thè resulting book captures thè spirit and fun of computing 
exemplified by SICP, but better suited to an introductory course for students 
with no previous background while covering many topics not included in SICP 
such as languages, complexity analysis, objects, and computability. Although 
this book is designed around a one semester introductory course, it should also 
be suitable for self-study students and for people with substantial programming 
experience but without similar computer Science knowledge. 


I am indebted to many people who helped develop this course and book. West- 
ley Weimer was thè first person to teach using something resembling this book, 
and his thorough and insightful feedback led to improvements throughout. Greg 
Humphreys, Paul Reynolds, and Mark Sherriff have also taught versions of this 
course, and contributed to its development. I am thankful to all of thè Assis- 
tami Coaches over thè years, especially Sarah Bergkuist (2004), Andrew Connors 
(2004), Rachel Dada (2003), Paul DiOrio (2009), Kinga Dobolyi (2007), Jon Erd- 
man (2002), Ethan Fast (2009), David Faulkner (2005), Jacques Fournier (2003), 
Richard Hsu (2007), Rachel Lathbury (2009), Michael Lew (2009), Stephen Liang 
(2002), Dan Marcus (2007), Rachel Rater (2009), Spencer Stockdale (2003), Dan 
Upton (2005), Portman Wills (2002), Katie Winstanley (2003 and 2004), and Re- 
becca Zapfel (2009) . William Aiello, Anna Chefter, Chris Frost, Jonathan Grier, 
Thad Hughes, AJan Kay, Tim Koogle, Jerry McGann, Gary McGraw, Radhika Nag- 
pal, Shawn O’Hargan, Mike Peck, and Judith Shatin also made important con- 
tributions to thè class and book. 

My deepest thanks are to my wife, Nora, who is a Constant source of inspiration, 
support, and wonder. 

Finally, my thanks to all past, present, and future students who use this book, 
without whom it would have no purpose. 

Happy Computing! 


David Evans 
Charlottesville, Virginia 
August 2011 
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Computing 


In their capacity as a tool, computers will be but a ripple on thè surface ofour 
culture. In their capacity as intellectual challenge, they are without precedent in 
thè cultural history ofmankind, 

Edsger Dijkstra, 1972 Turing Award Lecture 


The first millinn years of hominid history produced tools to amplify, and later 
mechanize, our physical abilities to enable us to move faster, reach higher, and 
hit harder. We have developed tools that amplify physical force by thè trillions 
and increase thè speeds at which we can travel by thè thousands. 

Tools that amplify intellectual abilities are much rarer. While some animals have 
developed tools to amplify their physical abilities, only humans have developed 
tools to substantially amplify our intellectual abilities and it is those advances 
that have enabled humans to dominate thè planet. The first key intellect am- 
plifier was language. Language provided thè ability to transmit our thoughts to 
others, as well as to use our own minds more effectively. The next key intellect 
amplifier was writing, which enabled thè Storage and transmission of thoughts 
over time and distance. 


Computing is thè ultimate mental amplifier — computers can mechanize any in- 
tellectual activity we can imagine. Automatic computing radically changes how 
humans solve problems, and even thè kinds of problems we can imagine solv- 
ing. Computing has changed thè world more than any other invention of thè 
past hundred years, and has come to pervade nearly all human endeavors. Yet, 
we are just at thè beginning of thè computing revolution; today’s computing of- 
fers just a glimpse of thè potential impact of computing. 


There are two reasons why everyone should study computing: 

1. Nearly all of thè most exciting and important technologies, arts, and Sci- 
ences of today and tomorrow are driven by computing. 

2. Understanding computing illuminates deep insights and questions into 
thè nature of our minds, our culture, and our universe. 

Anyone who has submitted a query to Google, watched Toy Story, had LASIK 
eye surgery, used a smartphone, seen a Cirque Du Soleil show, shopped with a 
credit card, or microwaved a pizza should be convinced of thè first reason. None 
of these would be possible without thè tremendous advances in computing over 
thè past half century. 

Although this hook will touch on on some exciting applications of computing, 
our primary focus is on thè second reason, which may seem more surprising. 


It may be trite that 
you have to be able 
to read in order to 
fili outforms at thè 
DMV, but that’s not 
why we teach 
children to read. We 
teach them to read 
for thè higher 
purpose ofallowing 
them access to 
beautiful and 
meaningful ideas. 
Paul Lockhart, 
Lockhart’s Lament 
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1.1. Processes, Procedures, andComputers 


Information 

processes 


procedure 


algorithm 


A mathematician is 
a machine for 
turning coffee into 
theorems. 
Attributed to Paul 
Erdòs 


Computing changes how we think about problems and how we understand thè 
world. The goal of this book is to teach you that new way of thinking. 

1.1 Processes, Procedures, and Computers 

Computer Science is thè study of information processes. A process is a sequence 
of steps. Each step changes thè state of thè world in some small way, and thè 
result of all thè steps produces some goal state. For example, baking a cake, 
mailing a letter, and planting a tree are all processes. Because they involve phys- 
ical things like sugar and dirt, however, they are not pure information processes. 
Computer Science focuses on processes that involve abstract information rather 
than physical things. 

The boundaries between thè physical world and pure information processes, 
however, are often fuzzy. Reai computers operate in thè physical world: they 
obtain input through physical means (e.g., a user pressing a key on a keyboard 
that produces an electrical impulse), and produce physical outputs (e.g., an im- 
age displayed on a screen). By focusing on abstract information, instead of thè 
physical ways of representing and manipulating information, we simplify com- 
putation to its essence to better enable understanding and reasoning. 

A procedure is a description of a process. A simple process can be described 
just by listing thè steps. The list of steps is thè procedure; thè act of following 
them is thè process. A procedure that can be followed without any thought is 
called a mechanical procedure. An algorithm is a mechanical procedure that is 
guaranteed to eventually finish. 

For example, here is a procedure for making coffee, adapted from thè actual 
directions that come with a major coffeemaker: 

1. Lift and open thè coffeemaker lid. 

2. Place a basket-type filter into thè filter basket. 

3. Add thè desired amount of coffee and shake to level thè coffee. 

4. Fili thè decanter with cold, fresh water to thè desired capacity. 

5. Pour thè water into thè water reservoir. 

6. Close thè lid. 

7. Place thè empty decanter on thè warming piate. 

8. Press thè ON button. 

Describing processes by just listing steps like this has many limitations. First, 
naturai languages are very imprecise and ambiguous. Following thè steps cor- 
rectly requires knowing lots of unstated assumptions. For example, step three 
assumes thè operator understands thè difference between coffee grounds and 
finished coffee, and can infer that this use of “coffee” refers to coffee grounds 
since thè end goal of this process is to make drinkable coffee. Other steps as- 
sume thè coffeemaker is plugged in and sitting on a fiat surface. 

One could, of course, add lots more details to our procedure and make thè lan- 
guage more precise than this. Even when a lot of effort is put into writing pre- 
cisely and clearly, however, naturai languages such as English are inherently am- 
biguous. This is why thè United States tax code is 3.4 million words long, but 
lawyers can stili spend years arguing over what it really means. 

Another problem with this way of describing a procedure is that thè size of thè 
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description is proportional to thè number of steps in thè process. This is fine 
for simple processes that can be executed by humans in a reasonable amount 
of time, but thè processes we want to execute on computers involve trillions of 
steps. This means we need more efficient ways to describe them than just listing 
each step one-by-one. 

To program computers, we need tools that allow us to describe processes pre- 
cisely and succinctly. Since thè procedures are carried out by a machine, every 
step needs to be described; we cannot rely on thè operator having “common 
sense” (for example, to know how to fili thè coffeemaker with water without ex- 
plaining that water comes from a faucet, and how to turn thè faucet on) . Instead, 
we need mechanical procedures that can be followed without any thinking. 

A computer is a machine that can: 

1. Accept input. Input could be entered by a human typing at a keyboard, 
received over a network, or provided automatically by sensors attached to 
thè computer. 

2. Execute a mechanical procedure, that is, a procedure where each step can 
be executed without any thought. 

3. Produce output. Output could be data displayed to a human, but it could 
also be anything that effects thè world outside thè computer such as elec- 
trical signals that control how a device operates. 

Computers exist in a wide range of forms, and thousands of computers are hid- 
den in devices we use everyday but don’t think of as computers such as cars, 
phones, TVs, microwave ovens, and access cards. Our primary focus is on uni- 
vermi computers, which are computers that can perform all possible mechan- 
ical computations on discrete inputs except for practical limits on space and 
time. The next section explains what it discrete inputs means; Chapters 6 and 12 
explore more deeply what it means for a computer to be universal. 

1 .2 Measuring Computing Power 

For physical machines, we can compare thè power of different machines by 
measuring thè amount of mechanical work they can perform within a given 
amount of time. This power can be captured with units like horsepower and 
watt. Physical power is not a very useful measure of computing power, though, 
since thè amount of computing achieved for thè same amount of energy varies 
greatly. Energy is consumed when a computer operates, but consuming energy 
is not thè purpose of using a computer. 

Two properties that measure thè power of a computing machine are: 

1. How much information it can process? 

2. How fast can it process? 

We defer considering thè second property until Part II, but consider thè first 
question here. 

1.2.1 Information 

Informally, we use information to mean knowledge. But to understand informa- 
tion quantitatively, as something we can measure, we need a more precise way 
to think about information. 


computer 


A computer 
terminal is not 
some clunky old 
television with a 
typewriter in front 
ofit. It is an 
interface where thè 
mind and body can 
connect with thè 
universe and move 
bits ofit about. 
Douglas Adams 


information 
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1.2. Measuring Computing Power 


The way computer scientists measure information is based on how what is known 
changes as a result of obtaining thè information. The primary unit of informa- 
te tion is a bit. One bit of information halves thè amount of uncertainty. It is equiv- 
alent to answering a “yes” or “no” question, where either answer is equally likely 
beforehand. Before learning thè answer, there were two possibilities; after learn- 
ing thè answer, there is one. 

binary question We cali a question with two possible answers a binary question. Since a bit can 
have two possible values, we often represent thè values as 0 and 1 . 

For example, suppose we perform a fair coin toss but do not reveal thè result. 
Half of thè time, thè coin will land “heads”, and thè other half of thè time thè 
coin will land “tails”. Without knowing any more information, our chances of 
guessing thè correct answer are \ . One bit of information would be enough to 
convey either “heads” or “tails”; we can use 0 to represent “heads” and 1 to rep- 
resent “tails”. So, thè amount of information in a coin toss is one bit. 

Similarly, one bit can distinguish between thè values 0 and 1: 



Example 1.1: Dice 


How many bits of information are there in thè outcome of tossing a six-sided 
die? 

There are six equally likely possible outcomes, so without any more information 
we have a one in six chance of guessing thè correct value. One bit is not enough 
to identify thè actual number, since one bit can only distinguish between two 
values. We could use fìve binary questions like this: 



This is quite inefficient, though, since we need up to fìve questions to identify 
thè value (and on average, expect to need 3j questions). Can we identify thè 
value with fewer than 5 questions? 
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Our goal is to identify questions where thè “yes” and “no” answers are equally 
likely — that way, each answer provides thè most information possible. This is 
not thè case if we start with, “Is thè value 6?”, since that answer is expected to be 
“yes” only one time in six. Instead, we should start with a question like, “Is thè 
value at least 4?”. Here, we expect thè answer to be “yes” one half of thè time, 
and thè “yes” and “no” answers are equally likely. If thè answer is “yes”, we know 
thè result is 4, 5, or 6. With two more bits, we can distinguish between these 
three values (note that two bits is actually enough to distinguish among four 
different values, so some information is wasted here) . Similarly, if thè answer 
to thè first question is no, we know thè result is 1, 2, or 3. We need two more 
bits to distinguish which of thè three values it is. Thus, with three bits, we can 
distinguish all six possible outcomes. 


No Yes 



Three bits can convey more information that just six possible outcomes, how- 
ever. In thè binary question tree, there are some questions where thè answer 
is not equally likely to be “yes” and “no” (for example, we expect thè answer to 
“Is thè value 3?” to be “yes” only one out of three times). Hence, we are not 
obtaining a full bit of information with each question. 

Each bit doubles thè number of possibilities we can distinguish, so with three 
bits we can distinguish between 2 * 2 * 2 — 8 possibilities. In generai, with ri bits, 
we can distinguish between 2" possibilities. Conversely, distinguishing among k 
possible values requires log 2 k bits. The logarithm is defìned such that if a — b c 
then log,, a — c. Since each bit has two possibilities, we use thè logarithm base 
2 to determine thè number of bits needed to distinguish among a set of distinct 
possibilities. For our six-sided die, log 2 6 « 2.58, so we need approximately 2.58 
binary questions. But, questions are discrete: we can’t ask 0.58 of a question, so 
we need to use three binary questions. 


Trees. Figure 1.1 depicts a structure of binary questions for distinguishing 
among eight values. We cali this structure a binary tree. We will see many useful 
applications of tree-like structures in this hook. 

Computer scientists draw trees upside down. The root is thè top of thè tree, and 
thè leaves are thè numbers at thè bottom (0, 1 , 2, . . ., 7). There is a unique path 
from thè root of thè tree to each leaf. Thus, we can describe each of thè eight 


logarithm 


binary tree 
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1.2. Measuring Computing Power 


possible values using thè answers to thè questions down thè tree. For example, 
if thè answers are “No”, “No”, and “No”, we reach thè leaf 0; if thè answers are 
“Yes”, “No”, “Yes”, we reach thè leaf 5. Since there are no more than two possible 
answers for each node, we cali this a binaiy tree. 

We can describe any non-negative integer using bits in this way, by just adding 
additional levels to thè tree. For example, if we wanted to distinguish between 
16 possible numbers, we would add a new question, “Is is >= 8?” to thè top 
of thè tree. If thè answer is “No”, we use thè tree in Figure 1.1 to distinguish 
numbers between 0 and 7. If thè answer is “Yes”, we use a tree similar to thè one 
in Figure 1.1, but add 8 to each of thè numbers in thè questions and thè leaves. 

depth The depili of a tree is thè length of thè longest path from thè root to any leaf. The 
example tree has depth three. A binary tree of depth d can distinguish up to 2 d 
different values. 


w w 


No Yes 


No Yes 


# * # * 


No Yes 


No Yes 


No Yes 


No Yes 


Figure 1.1. Using three bits to distinguish eight possible values. 


Units of Information. One byte is defined as eight bits. Hence, one byte of 
information corresponds to eight binary questions, and can distinguish among 
2 8 (256) different values. For larger amounts of information, we use metric pre- 
fixes, but instead of scaling by factors of 1 000 they scale by factors of 2 10 (1 024) . 
Hence, one kilobyte is 1024 bytes; one megabyte is 2 20 (approximately one mil- 
lion) bytes; one gigabyte is 2 30 (approximately one billion) bytes; and one ter- 
abyte is 2 40 (approximately one trillion) bytes. 


Exercise 1.1. Draw a binary tree with thè minimum possible depth to: 

a. Distinguish among thè numbers 0, 1, 2 , ... , 15. 

b. Distinguish among thè 12 months of thè year. 
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Exercise 1.2. How many bits are needed: 

a. To uniquely identify any currently living human? 

b. To uniquely identify any human who ever lived? 

c. To identify any location on Earth within one square centimeter? 

d. To uniquely identify any atom in thè observable universe? 

Exercise 1.3. The examples all use binary questions for which there are two 
possible answers. Suppose instead of basing our decisions on bits, we based it 
on trits where one trit can distinguish between three equally likely values. For 
each trit, we can ask a ternary question (a question with three possible answers) . 


a. How many trits are needed to distinguish among eight possible values? (A 
convincing answer would show a ternary tree with thè questions and answers 
for each node, and argue why it is not possible to distinguish all thè values 
with a tree of lesser depth.) 

b. [*] Devise a generai formula for converting between bits and trits. How many 
trits does it require to describe b bits of information? 


Exploration 1.1: Guessing Numbers 


The guess-a-number game starts with one player (thè chooser) picking a num- 
ber between 1 and 100 (inclusive) and secretly writing it down. The other player 
(thè guesser ) attempts to guess thè number. After each guess, thè chooser re- 
sponds with “correct” (thè guesser guessed thè number and thè game is over), 
“higher” (thè actual number is higher than thè guess), or ‘Tower” (thè actual 
number is lower than thè guess). 

a. Explain why thè guesser can receive slightly more than one bit of information 
for each response. 

b. Assuming thè chooser picks thè number randomly (that is, all values between 
1 and 100 are equally likely), what are thè best frrst guesses? Explain why 
these guesses are better than any other guess. (Hint: there are two equally 
good first guesses.) 

c. What is thè maximum number of guesses thè second player should need to 
always find thè number? 

d. What is thè average number of guesses needed (assuming thè chooser picks 
thè number randomly as before)? 

e. [*] Suppose instead of picking randomly, thè chooser picks thè number with 
thè goal of maximizing thè number of guesses thè second player will need. 
What number should she pick? 

f. [**] How should thè guesser adjust her strategy if she knows thè chooser is 
picking adversarially? 

g. [**] What are thè best strategies for both players in thè adversarial guess-a- 
number game where chooser’s goal is to pick a starting number that maxi- 
mizes thè number of guesses thè guesser needs, and thè guesser’s goal is to 
guess thè number using as few guesses as possible. 
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20Q Game 

Image from ThinkGeek 


binary number 
System 

There are only 10 
types ofpeople in 
thè world: 
those who under- 
stand binary, 
and those who don’t. 
Infamous T-Shirt 


Exploration 1.2: Twenty Questions 


The two-player game twenty questions starts with thè first player (thè answerer) 
thinking of an object, and declaring if thè object is an animai, vegetable, or min- 
erai (meant to include all non-living things). After this, thè second player (thè 
questioner), asks binary questions to try and guess thè object thè first player 
thought of. The first player answers each question “yes” or “no”. The website 
http://www.20q.net/ offers a web-based twenty questions game where a human 
acts as thè answerer and thè computer as thè questioner. The game is also sold 
as a $10 stand-alone toy (shown in thè picture). 

a. How many different objects can be distinguished by a perfect questioner for 
thè standard twenty questions game? 

b. What does it mean for thè questioner to play perfectly? 

c. Try playing thè 20Q game at http://www.20q.net. Did it guess your item? 

d. Instead of just “yes” and “no”, thè 20Q game offers four different answers: 
“Yes”, “No”, “Sometimes”, and “Unknown”. (The website version of thè game 
also has “Probably”, “Irrelevant”, and “Doubtful”) If all four answers were 
equally likely (and meaningful), how many items could be distinguished in 
20 questions? 

e. For an Animai, thè first question 20Q sometimes asks is “Does it jump?” (20Q 
randomly selected from a few different first questions). Is this a good first 
question? 

f. [*] How many items do you think 20Q has data for? 

g. [**] Speculate on how 20Q could build up its database. 


1.2.2 Representing Data 

We can use sequences of bits to represent many kinds of data. All we need to do 
is think of thè right binary questions for which thè bits give answers that allow 
us to represent each possible value. Next, we provide examples showing how 
bits can be used to represent numbers, text, and pictures. 

Numbers. In thè previous section, we identified a number using a tree where 
each node asks a binary question and thè branches correspond to thè “Yes” and 
“No” answers. A more compact way of writing down our decisions following thè 
tree is to use 0 to encode a “No” answer, and 1 to encode a “Yes” answer and 
describe a path to a leaf by a sequence of Os and 1 s — thè “No”, “No”, “No” path to 
0 is encoded as 000, and thè “Yes”, “No”, “Yes” path to 5 is encoded as 1 01 . This is 
known as thè binary number System. Whereas thè decimai number System uses 
ten as its base (there are ten decimai digits, and thè positional values increase 
as powers of ten), thè binary System uses two as its base (there are two binary 
digits, and thè positional values increase as powers of two) . 

For example, thè binary number 10010110 represents thè decimai value 1 50: 


Binary: 

1 

0 

0 

1 

0 

1 

1 

0 

Value: 

2 7 

2 b 

2 b 

2 4 

2 3 

2 1 

2 1 

2° 

Decimai Value: 

128 

64 

32 

16 

8 

4 

2 

1 
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As in thè decimai number System, thè value of each binary digit depends on its 
position. 

By using more bits, we can represent larger numbers. With enough bits, we can 
represent any naturai number this way. The more bits we have, thè larger thè set 
of possible numbers we can represent. As we saw with thè binary decision trees, 
n bits can be used to represent 2" different numbers. 

Discrete Values. We can use a finite sequence of bits to describe any value that 
is selected from a countable set of possible values. A set is countable if there is a countable 
way to assign a unique naturai number to each element of thè set. All finite sets 
are countable. Some, but not all, infinite sets are countable. For example, there 
appear to be more integers than there are naturai numbers since for each naturai 
number, n, there are two corresponding integers, n and —n. But, thè integers are 
infact countable. We can enumerate thè integers as: 0,1, —1,2, -2, 3, -3,4, -4, . . . 
and assign a unique naturai number to each integer in turn. 

Other sets, such as thè reai numbers, are uncountable. Georg Cantor proved 

this using a technique known as diagonalization. Suppose thè reai numbers are diagonalization 

enumerable. This means we could list all thè reai numbers in order, so we could 

assign a unique integer to each number. For example, considering just thè reai 

numbers between 0 and 1, our enumeration might be: 


1 

2 

3 

4 


00000000000000 . . . 

25000000000000 . . . 

33333333333333 . . . 

6666666666666 . . . 


57236 


141592653589793 . . . 


Cantor proved by contradiction that there is no way to enumerate all thè reai 
numbers. The trick is to produce a new reai number that is not part of thè enu- 
meration. We can do this by constructing a number whose first digit is different 
from thè first digit of thè first number, whose second digit is different from thè 
second digit of thè second number, etc. For thè example enumeration above, we 
might choose .1468 .... 

The k th digit of thè constructed number is different from thè k th digit of thè num- 
ber k in thè enumeration. Since thè constructed number differs in at least one 
digit from every enumerated number, it does not match any of thè enumerated 
numbers exactly. Thus, there is a reai number that is not included in thè enu- 
meration list, and it is impossible to enumerate all thè reai numbers. 1 

Digital computers 2 operate on inputs that are discrete values. Continuous val- 
ues, such as reai numbers, can only be approximated by computers. Next, we 

^ert readers should be worried that this isn’t quite correct since thè resulting number may be a 
different way to represent thè same reai number (for example, .1999999999999 . . . = .20000000000 . . . 
even though they differ in each digit). This technical problem can be fìxed by placing some restric- 
tions on how thè modifìed digits are chosen to avoid infinite repetitions. 

2 This is, indeed, part of thè definition of a digitai computer. An analog computer operates on 
continuous values. In Chapter 6, we explain more of thè inner workings of a computer and why 
nearly all computers today are digitai. We use computer to mean a digitai computer in this book. 
The property that there are more reai numbers than naturai numbers has important implications 
for what can and cannot be computed, which we return to in Chapter 12. 
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consider how two types of data, text and images, can be represented by com- 
puters. The first type, text, is discrete and can be represented exactly; images 
are continuous, and can only be represented approximately. 

Text. The set of all possible sequences of characters is countable. One way to 
see this is to observe that we could give each possible text fragment a unique 
number, and then use that number to identify thè item. For example we could 
enumerate all texts alphabetically by length (here, we limit thè characters to low- 
ercase letters): a, b, c, . . ., z, aa, ab, . . ., az, ba, . . ., zz, aaa, . . . 

Since we have seen that we can represent all thè naturai numbers with a se- 
quence of bits, so once we have thè mapping between each item in thè set and 
a unique naturai number, we can represent all of thè items in thè set. For thè 
representation to be useful, though, we usually need a way to construct thè cor- 
responding number for any item directly. 

So, instead of enumerating a mapping between all possible character sequences 
and thè naturai numbers, we need a process for converting any text to a unique 
number that represents that text. Suppose we limit our text to characters in 
thè standard English alphabet. If we include lower-case letters (26), upper-case 
letters (26), and punctuation (space, comma, period, newline, semi-colon), we 
have 57 different symbols to represent. We can assign a unique number to each 
Symbol, and encode thè corresponding number with six bits (this leaves seven 
values unused since six bits can distinguish 64 values). For example, we could 
encode using thè mapping shown in Table 1.1. The hrst bit answers thè ques- 
tioni “Is it an uppercase letter after F or a special character?”. When thè hrst bit 
is 0, thè second bit answers thè questioni “Is it after p?”. 


a 

b 

000000 

000001 


A 

B 

011010 

011011 


space 

110100 

110101 



} 

c 

000010 


c 

011100 



110110 

d 

000011 


F 

011111 


newline 

110111 

111000 

P 

001111 


G 

100000 


unused 

111001 

q 

010000 


Y 

110010 


unused 

111110 

Z 

011001 


Z 

110011 


unused 

111111 


Table 1.1. Encoding characters using bits. 

This is one way to encode thè alphabet, but not thè one typically used by computers. 
One commonly used encoding known as ASCII (thè American Standard Code for Infor- 
mation Interchange) uses seven bits so that 128 different symbols can be encoded. The 
extra symbols are used to encode more special characters. 


Once we have a way of mapping each individuai letter to a hxed-length bit se- 
quence, we could write down any sequence of letters by just concatenating thè 
bits encoding each letter. So, thè text CS is encoded as 011100 101100. We 
could write down text of length n that is written in thè 57-symbol alphabet using 
this encoding using 6 n bits. To convert thè number back into text, just invert thè 
mapping by replacing each group of six bits with thè corresponding letter. 

Rich Data. We can also use bit sequences to represent complex data like pic- 
tures, movies, and audio recordings. First, consider a simple black and white 
picture: 
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0 0 


Sirice thè picture is divided into discrete squares known as pixels, we could en- pixel 
code this as a sequence of bits by using one bit to encode thè color of each pixel 
(for example, using 1 to represent black, and 0 to represent white). This image 
is 16x16, so has 256 pixels total. We could represent thè image using a sequence 
of 256 bits (starting from thè top left corner) : 


0000011111100000 

0000100000010000 

0011000000001100 

0010000000000100 

0000011111100000 

What about complex pictures that are not divided into discrete squares or a fixed 
number of colors, like Van Gogh’s Starry Niglift 



Different wavelengths of electromagnetic radiation have different colors. For 
example, light with wavelengths between 625 and 730 nanometers appears red. 
But, each wavelength of light has a slightly different color; for example, light with 
wavelength 650 nanometers would be a different color (albeit imperceptible to 
humans) from light of wavelength 650.0000001 nanometers. There are arguably 
infinitely many different colors, corresponding to different wavelengths of visi- 
tile light. 3 Since thè colors are continuous and not discrete, there is no way to 
map each color to a unique, finite bit sequence. 

3 Whether there are actually infinitely many different colors Comes down to thè question of 
whether thè space-time of thè universe is continuous or discrete. Certainly in our common per- 
ception it seems to be continuous — we can imagine dividing any length into two shorter lengths. In 
reality, this may not be thè case at extremely tiny scales. It is not known if time can continue to be 
subdivided below 10~ 40 of a second. 
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On thè other hand, thè human eye and brain have limits. We cannot actu- 
ally perceive infìnitely many different colors; at some point thè wavelengths 
are dose enough that we cannot distinguish them. Ability to distinguish colors 
varies, but most humans can perceive only a few million different colors. The set 
of colors that can be distinguished by a typical human is finite; any finite set is 
countable, so we can map each distinguishable color to a unique bit sequence. 

A common way to represent color is to break it into its three primary compo- 
nents (red, green, and blue), and record thè intensity of each component. The 
more bits available to represent a color, thè more different colors that can be 
represented. 

Thus, we can represent a picture by recording thè approximate color at each 
point. If space in thè universe is continuous, there are infìnitely many points. 
But, as with color, once thè points get smaller than a certain size they are im- 
perceptible. We can approximate thè picture by dividing thè canvas into small 
regions and sampling thè average color of each region. The smaller thè sample 
regions, thè more bits we will have and thè more detail that will be visible in thè 
image. With enough bits to represent color, and enough sample points, we can 
represent any image as a sequence of bits. 

Summary. We can use sequences of bits to represent any naturai number ex- 
actly, and hence, represent any member of a countable set using a sequence of 
bits. The more bits we use thè more different values that can be represented; 
with n bits we can represent 2" different values. 

We can also use sequences of bits to represent rich data like images, audio, and 
video. Since thè world we are trying to represent is continuous there are in- 
finitely many possible values, and we cannot represent these objects exactly 
with any finite sequence of bits. However, since human perception is limited, 
with enough bits we can represent any of these adequately well. Finding ways 
to represent data that are both efficient and easy to manipulate and interpret is 
a Constant challenge in computing. Manipulating sequences of bits is awkward, 
so we need ways of thinking about bit-level representations of data at higher 
levels of abstraction. Chapter 5 focuses on ways to manage complex data. 

1.2.3 Growth of Computing Power 

The number of bits a computer can store gives an upper limit on thè amount of 
information it can process. Looking at thè number of bits different computers 
can store over time gives us a rough indication of how computing power has 
increased. Here, we consider two machines: thè Apollo Guidance Computer 
and a modern laptop. 

The Apollo Guidance Computer was developed in thè early 1960s to control thè 
flight Systems of thè Apollo spacecraft. It might be considered thè first personal 
computer, since it was designed to be used in real-time by a single operator (an 
astronaut in thè Apollo capsule) . Most earlier computers required a full room, 
and were far too expensive to be devoted to a single user; instead, they pro- 
cessed jobs submitted by many users in turn. Since thè Apollo Guidance Com- 
puter was designed to fit in thè Apollo capsule, it needed to be small and light. 
Its volume was about a cubie foot and it weighed 70 pounds. The AGC was 
agc User interface thè first computer built using integrated circuits, miniature electronic circuits 
that can perform simple logicai operations such as performing thè logicai and 
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of two values. The AGC used about 4000 integrateci circuits, each one being able 
to perform a single logicai operation and costing $1000. The AGC consumed a 
significant fraction of all integrated circuits produced in thè mid-1960s, and thè 
project spurred thè growth of thè integrated Circuit industry. 


The AGC had 552 960 bits of memory (of which only 61 440 bits were modihable, 
thè rest were frxed). The smallest USB flash memory you can buy today (from 
SanDisk in December 2008) is thè 1 gigabyte Cruzer for $9.99; 1 gigabyte (GB) 
is 2 30 bytes or approximately 8.6 billion bits, about 140 000 times thè amount of 
memory in thè AGC (and all of thè Cruzer memory is modifìable). A typical low- 
end laptop today has 2 gigabytes of RAM (fast memory dose to thè processor 
that loses its state when thè machine is turned off) and 250 gigabytes of hard 
disk memory (slow memory that persists when thè machine is turned off); for 
under $600 today we get a computer with over 4 million times thè amount of 
memory thè AGC had. 

Improving by a factor of 4 million corresponds to doubling just over 22 times. 
The amount of computing power approximately doubled every two years be- 
tween thè AGC in thè early 1960s and a modern laptop today (2009). This prop- 
erty of exponential improvement in computing power is known as Moore’s Law. 
Gordon Moore, a co-founder of Intel, observed in 1965 than thè number of com- 
ponents that can be built in integrated circuits for thè same cost was approxi- 
mately doubling every year (revisions to Moore’s observation have put thè dou- 
bling rate at approximately 18 months instead of one year). This progress has 
been driven by thè growth of thè computing industry, increasing thè resources 
available for designing integrated circuits. Another driver is that today’s tech- 
nology is used to design thè next technology generation. Improvement in com- 
puting power has followed this exponential growth remarkably closely over thè 
past 40 years, although there is no law that this growth must continue forever. 


Moore’s law is a 
violation of 
Murphy’s law. 
Everything gets 
better and better. 
Gordon Moore 


Although our comparison between thè AGC and a modern laptop shows an im- 
pressive factor of 4 million improvement, it is much slower than Moore’s law 
would suggest. Instead of 22 doublings in power since 1963, there should have 
been 30 doublings (using thè 18 month doubling rate). This would produce an 
improvement of one billion times instead of just 4 million. The reason is our 
comparison is very unequal relative to cost: thè AGC was thè world’s most ex- 
pensive small computer of its time, reflecting many millions of dollars of gov- 
ernment funding. Computing power available for similar funding today is well 
over a billion times more powerful than thè AGC. 


1 .3 Science, Engineering, and thè Liberal Arts 

Much ink and many bits have been spent debating whether computer Science 
is an art, an engineering discipline, or a Science. The confusion stems from thè 
nature of computing as a new field that does not fit well into existing silos. In 
fact, computer Science fìts into all three kingdoms, and it is useful to approach 
computing from all three perspectives. 

Science. Traditional Science is about understanding nature through observa- 
tion. The goal of Science is to develop generai and predictive theories that allow 
us to understand aspects of nature deeply enough to make accurate quantitative 
predications. For example, Newton’s law of universal gravitation makes predic- 
tions about how masses will move. The more generai a theory is thè better. A key, 
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as yet unachieved, goal of Science is to find a universal law that can describe all 
physical behavior at scales from thè smallest subparticle to thè entire universe, 
and all thè bosons, muons, dark matter, black holes, and galaxies in between. 
Science deals with reai things (like bowling balls, planets, and electrons) and at- 
tempts to make progress toward theories that predict increasingly precisely how 
these reai things will behave in different situations. 


Computer Science focuses on artificial things like numbers, graphs, functions, 
and lists. Instead of dealing with physical things in thè reai world, computer sci- 
enee concerns abstract things in a Virtual world. The numbers we use in compu- 
tations often represent properties of physical things in thè reai world, and with 
enough bits we can model reai things with arbitrary precision. But, since our fo- 
cus is on abstract, artificial things rather than physical things, computer Science 
is not a traditional naturai Science but a more abstract field like mathematics. 
Like mathematics, computing is an essential tool for modern Science, but when 
we study computing on artificial things it is not a naturai Science itself. 

In a deeper sense, computing pervades all of nature. A long term goal of com- 
puter Science is to develop theories that explain how nature computes. One ex- 
ample of computing in nature comes from biology. Complex life exists because 
nature can perform sophisticated computing. People sometimes describe DNA 
as a “blueprint”, but it is really much better thought of as a program. Whereas 
a blueprint describes what a building should be when it is fmished, giving thè 
dimensions of walls and how they fìt together, thè DNA of an organism encodes 
a process for growing that organism. A human genome is not a blueprint that 
describes thè body pian of a human, it is a program that turns a single celi into 
a complex human given thè appropriate environment. The process of evolution 
(which itself is an information process) produces new programs, and hence new 
species, through thè process of naturai selection on mutated DNA sequences. 
Understanding how both these processes work is one of thè most interesting 
and important open scientifìc questions, and it involves deep questions in com- 
puter Science, as well as biology, chemistry, and physics. 


Scientists study thè 
world as it is; 
engineers create thè 
world that never 
has been. 
Theodore von Karman 


The questions we consider in this hook focus on thè question of what can and 
cannot be computed. This is both a theoretical question (what can be computed 
by a given theoretical model of a computer) and a pragmatic one (what can be 
computed by physical machines we can build today, as well as by anything pos- 
sibile in our universe). 

Engineering. Engineering is about making useful things. Engineering is of- 
ten distinguished from crafts in that engineers use scientifìc principles to create 
their designs, and focus on designing under practical constraints. As William 
Wulf and George Fisher put it : 4 


Whereas Science is analytic in that it strives to understand nature, or what 
is, engineering is synthetic in that it strives to create. Our oum favorite de- 
scription of what engineers do is “design under constraint". Engineering is 
creativity constrained by nature, bycost, by concerns ofsafety, environmen- 
tal impact, ergonomics, reliability, manufacturability, maintainability- 
the whole long list of such “ilities”. To be sure, thè realities of nature is one 
ofthe constraint sets we work under, but it is far from thè only one, it is 


4 William Wulf and George Fisher, A Makeover for Engineering Education, Issues in Science and 
Technology, Spring 2002 (http://www.issues.Org/1 8.3/p_wulf.html). 
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seldom thè hardest one, and almost never thè limiting one. 

Computer scientists do not typically face thè naturai constraints faced by civil 
and mechanical engineers — computer programs are massless and not exposed 
to thè weather, so programmers do not face thè kinds of physical constraints like 
gravity that impose limits on bridge designers. As we saw from thè Apollo Guid- 
ance Computer comparison, practical constraints on computing power change 
rapidly — thè one billion times improvement in computing power is unlike any 
change in physical materials 5 . Although we may need to worry about manufac- 
turability and maintainability of Storage media (such as thè disk we use to sto re 
a program), our focus as computer scientists is on thè abstract bits themselves, 
not how they are stored. 

Computer scientists, however, do face many constraints. A primary constraint 
is thè capacity of thè human mind — there is a limit to how much information a 
human can keep in mind at one time. As computing Systems get more complex, 
there is no way for a human to understand thè entire System at once. To build 
complex Systems, we need techniques for managing complexity. The primary 
tool computer scientists use to manage complexity is abstraction. Abstraction is 
a way of giving a name to something in a way that allows us to hide unnecessary 
details. By using carefully designed abstractions, we can construct complex Sys- 
tems with reliable properties while limiting thè amount of information a human 
designer needs to keep in mind at any one time. 

Liberal Arts. The notion of thè liberal arts emerged during thè middle ages to 
distinguish education for thè purpose of expanding thè intellects of free people 
from thè illiberal arts such as medicine and carpentry that were pursued for 
economie purposes. The liberal arts were intended for people who did not need 
to learn an art to make a living, but instead had thè luxury to pursue purely 
intellectual activities for their own sake. The traditional seven liberal arts started 
with thè Trivium (three roads), focused on language : 6 

• Grammar — “thè art of inventing symbols and combining them to express 
thought” 

• Rhetoric — “thè art of communicating thought from one mind to another, 
thè adaptation of language to circumstance” 

• Logic — “thè art of thinking” 

The Trivium was followed by thè Quadrivium, focused on numbers: 

• Arithmetic — “theory of number” 

• Geometry — “theory of space” 

• Music — “application of thè theory of number” 

• Astronomy — “application of thè theory of space” 

All of these have strong connections to computer Science, and we will touch on 
each of them to some degree in this hook. 

Language is essential to computing since we use thè tools of language to de- 
scritte information processes. The next chapter discusses thè structure of lan- 
guage and throughout this hook we consider how to effìciently use and combine 

5 For example, thè highest strength density material available today, carbon nanotubes, are per- 
haps 300 times stronger than thè best material available 50 years ago. 

6 The quotes defining each liberal art are from Miriam Joseph (edited by Marguerite McGlinn), 
The Trivium: The Liberal Arts of Logic, Grammar, and Rhetoric, Paul Dry Books, 2002. 


abstraction 


I must study politics 
and war that my 
sons may have 
liberty to study 
mathematics and 
philosophy. Mysons 
ought to study 
mathematics and 
philosophy, 
geography, naturai 
histoiy, naval 
architecture, 
navigation, 
commerce, and 
agriculture, in order 
to give their 
children a right to 
study painting, 
poetiy, music, 
architecture, 
statuary, tapestry, 
and porcelain. 

John Adams, 1780 


16 


1.4. Summary and Roadmap 


symbols to express meanings. Rhetoric encompasses communicating thoughts 
between minds. In computing, we are not typically communicating directly be- 
tween minds, but we see many forms of communication between entities: in- 
terfaces between components of a program, as well as protocols used to enable 
multiple computing Systems to communicate (for example, thè HTTP protocol 
defines how a web browser and web server interact), and communication be- 
tween computer programs and human users. The primary tool for understand- 
ing what computer programs mean, and hence, for constructing programs with 
particular meanings, is logie. Hence, thè traditional trivium liberal arts of lan- 
guage and logie permeate computer Science. 

The connections between computing and thè quadrivium arts are also perva- 
sive. We have already seen how computers use sequences of bits to represent 
numbers. Chapter 6 examines how machines can perform basic arithmetic op- 
erations. Geometry is essential for computer graphics, and graph theory is also 
important for computer networking. The harmonic structures in music have 
strong connections to thè recursive definitions introduced in Chapter 4 and re- 
curring throughout this book. 7 Unlike thè other six liberal arts, astronomy is not 
directly connected to computing, but computing is an essential tool for doing 
modem astronomy. 

Although learning about computing qualifies as an illiberal art (that is, it can 
have substantial economie benefits for those who learn it well), computer sci- 
enee also covers at least six of thè traditional seven liberal arts. 

1 .4 Summary and Roadmap 

Computer scientists think about problems differently. When confronted with a 
problem, a computer scientist does not just attempt to solve it. Instead, com- 
puter scientists think about a problem as a mapping between its inputs and de- 
sired outputs, develop a systematic sequence of steps for solving thè problem 
for any possible input, and consider how thè number of steps required to solve 
thè problem scales as thè input size increases. 

The rest of this book presents a whirlwind introduction to computer Science. 
We do not cover any topics in great depth, but rather provide a broad picture 
of what computer Science is, how to think like a computer scientist, and how to 
solve problems. 

Part I: Defining Procedures. Part I focuses on how to define procedures that 
perform desired computations. The nature of thè computer forces Solutions to 
be expressed precisely in a language thè computer can interpret. This means a 
computer scientist needs to understand how languages work and exactly what 
phrases in a language mean. Naturai languages like English are too complex and 
inexact for this, so we need to invent and use new languages that are simpler, 
more structured, and less ambiguously defined than naturai languages. Chap- 
ter 2 focuses on language, and during thè course of this book we will use lan- 
guage to precisely describe processes and languages are interpreted. 

The computer frees humans from having to actually carry out thè steps needed 
to solve thè problem. Without complaint, boredom, or rebellion, it dutifully ex- 

7 See Douglas Hofstadter’s Godei, Escher, Bach for lots of interesting examples of connections be- 
tween computing and music. 
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ecutes thè exact steps thè program specihes. And it executes them at a remark- 
able rate — billions of simple steps in each second on a typical laptop. This 
changes not just thè time it takes to solve a problem, but qualitatively changes 
thè kinds of problems we can solve, and thè kinds of Solutions worth consid- 
ering. Problems like sequencing thè human genome, simulating thè global ch- 
inate, and making a photomosaic not only could not have been solved without 
computing, but perhaps could not have even been envisioned. Chapter 3 intro- 
duces programming, and Chapter 4 develops some techniques for constructing 
programs that solve problems. To represent more interesting problems, we need 
ways to manage more complex data. Chapter 5 concludes Part I by exploring 
ways to represent data and debne procedures that operate on complex data. 

Part II: Analyzing Procedures. Part II considers thè problem of estimating thè 
cost required to execute a procedure. This requires understanding how ma- 
chines can compute (Chapter 6), and mathematical tools for reasoning about 
how cost grows with thè size of thè inputs to a procedure (Chapter 7). Chapter 8 
provides some extended examples that apply these techniques. 

Part III: Improving Expressiveness. The techniques from Part I and II are suf- 
hcient for describing all computations. Our goal, however, it to be able to define 
concise, elegant, and efficient procedures for performing desired computations. 
Part III presents techniques that enable more expressive procedures. 

Part IV: The Limits of Computing. We hope that by thè end of Part III, readers 
will feel conhdent that they could program a computer to do just about any- 
thing. In Part IV, we consider thè question of what can and cannot be done by a 
mechanical computer. A large class of interesting problems cannot be solved by 
any computer, even with unlimited time and space. 

Themes. Much of thè hook will revolve around three very powerful ideas that 
are prevalent throughout computing: 

Recursive deflnitions. A recursive definition define a thing in terms of smaller 
instances of itself. A simple example is dehning your ancestors as (1) your par- 
ents, and (2) thè ancestors of your ancestors. Recursive definitions can debne 
an inbnitely large set with a small description. They also provide a powerful 
technique for solving problems by breaking a problem into solving a simple in- 
stance of thè problem and showing how to solve a larger instance of thè problem 
by using a solution to a smaller instance. We use recursive debnitions to debne 
inbnite languages in Chapter 2, to solve problems in Chapter 4, to build complex 
data structures in Chapter 5. In later chapters, we see how language interpreters 
themselves can be debned recursively. 

Universality. Computers are distinguished from other machines in that their be- 
havior can be changed by a program. Procedures themselves can be described 
using just bits, so we can write procedures that process procedures as inputs and 
that generate procedures as outputs. Considering procedures as data is both a 
powerful problem solving tool, and a useful way of thinking about thè power 
and fundamental limits of computing. We introduce thè use of procedures as 
inputs and outputs in Chapter 4, see how generated procedures can be pack- 
aged with state to model objects in Chapter 10. One of thè most fundamental 
results in computing is that any machine that can perform a few simple opera- 
tions is powerful enough to perform any computation, and in this deep sense, 
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all mechanical computers are equivalent. We introduce a model of computation 
in Chapter 6, and reason about thè limits of computation in Chapter 12. 

Abstraction. Abstraction is a way of hiding details by giving things names. We 
use abstraction to manage complexity. Good abstractions hide unnecessary de- 
tails so they can be used to build complex Systems without needing to under- 
stand all thè details of thè abstraction at once. We introduce procedural ab- 
straction in Chapter 4, data abstraction in Chapter 5, abstraction using objects 
in Chapter 10, and many other examples of abstraction throughout this book. 

Throughout this book, these three themes will recur recursively, universally, and 
abstractly as we explore thè art and Science of how to instruct computing ma- 
chines to perform useful tasks, reason about thè resources needed to execute a 
particular procedure, and understand thè fundamental and practical limits on 
what computers can do. 



Language 


Belittle! What an expression! It rnay be an elegant one in Virginia, and even perfectly intelligible; 

butfor our pari, all we cari do is to guess at its meaning. For sitarne, Mr. Jefferson! 

European Magazine and London Review, 1787 
(reviewing Thomas Jefferson’s Notes on thè State of Virginia) 

The most powerful tool we have for communication is language. This is true 
whether we are considering communication between two humans, between a 
human programmer and a computer, or between a network of computers. In 
computing, we use language to describe procedures and use machines to turn 
descriptions of procedures into executing processes. This chapter is about what 
language is, how language works, and ways to debne languages. 


2.1 Surface Forms and Meanings 

A language is a set of surface forms and meanings, and a mapping between thè language 
surface forms and their associated meanings. In thè earliest human languages, 
thè surface forms were sounds but surface forms can be anything that can be 
perceived by thè communicating parties such as drum beats, hand gestures, or 
pictures. 


A naturai language is a language spoken by humans, such as English or Swahili, naturai language 
Naturai languages are very complex since they have evolved over many thou- 
sands years of individuai and cultural interaction. We focus on designed lan- 
guages that are created by humans for some a specifìc purpose such as for ex- 
pressing procedures to be executed by computers. 


We focus on languages where thè surface forms are text. In a textual language, 
thè surface forms are linear sequences of characters. A string is a sequence of string 
zero or more characters. Each character is a symbol drawn from a finite set 
known as an alphabet. For English, thè alphabet is thè set {a,b,c,. ..,z) (for alphabet 
thè full language, capitai letters, numerals, and punctuation symbols are also 
needed). 

A simple communication System can be described using a table of surface forms 
and their associated meanings. For example, this table describes a communica- 
tion System between traffìc lights and drivers: 

Surface Form Meaning 
Green Go 

Yellow Caution 

Red Stop 
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Communication Systems involving humans are notoriously imprecise and sub- 
jective. A driver and a police officer may disagree on thè actual meaning of thè 
Yellow Symbol, and may even disagree on which Symbol is being transmitted by 
thè traffic light at a particular time. Communication Systems for computers de- 
mand precision: we want to know what our programs will do, so it is important 
that every step they make is understood precisely and unambiguously. 

The method of defining a communication System by listing a table of 

< Symbol , Meaning > 



pairs can work adequately only for trivial communication Systems. The number 
of possible meanings that can be expressed is limited by thè number of entries in 
thè table. It is impossible to express any new meaning since all meanings must 
already be listed in thè table! 

Languages and Infinity. A useful language must be able to express inflnitely 
many different meanings. Hence, there must be a way to generate new sur- 
face forms and guess their meanings (see Exercise 2.1). No finite representation, 
such as a printed table, can contain all thè surface forms and meanings in an 
infinite language. One way to generate infinitely large sets is to use repeating 
patterns. For example, most humans would interpret thè notation: “1, 2, 3, ... ” 
as thè set of all naturai numbers. We interpret thè “. . . ” as meaning keep doing 
thè same thing for ever. In this case, it means keep adding one to thè preced- 
ing number. Thus, with only a few numbers and symbols we can describe a set 
containing infinitely many numbers. As discussedin Section 1.2.1, thè language 
of thè naturai numbers is enough to encode all meanings in any countable set. 
But, finding a sensible mapping between most meanings and numbers is nearly 
impossible. The surface forms do not correspond closely enough to thè ideas we 
want to express to be a useful language. 


2.2 Language Construction 

To define more expressive infinite languages, we need a richer System for con- 
structing new surface forms and associated meanings. We need ways to de- 
scribe languages that allow us to define an infinitely large set of surface forms 
and meanings with a compact notation. The approach we use is to define a lan- 
guage by defining a set of rules that produce exactly thè set of surface forms in 
thè language. 

Components of Language. A language is composed of: 

• primitives — thè smallest units of meaning. 

• means of combination — rules for building new language elements by com- 
bining simpler ones. 

The primitives are thè smallest meaningful units (in naturai languages these are 
known as morphemes ). A primitive cannot be broken into smaller parts whose 
meanings can be combined to produce thè meaning of thè unit. The means 
of combination are rules for building words from primitives, and for building 
phrases and sentences from words. 

Since we have rules for producing new words not all words are primitives. For 
example, we can create a new word by adding antì- in front of an existing word. 



Chapter2. Language 


21 


The meaning of thè new word can be inferred as “against thè meaning of thè 
originai word”. Rules like this one mean anyone can invent a new word, and use 
it in communication in ways that will probably be understood by listeners who 
have never heard thè word before. 

For example, thè vexb freeze means to pass from a liquid state to a solid state; an- 
tifreeze is a substance designed to prevent freezing. English speakers who know 
thè meaning of freeze and ariti- could roughly guess thè meaning of antifreeze 
even if they have never heard thè word before. 1 

Primitives are thè smallest units of meaning, not based on thè surface forms. 
Both anti and freeze are primitive; they cannot be broken into smaller parts 
with meaning. We can break anti- into two syllables, or four letters, but those 
sub-components do not have meanings that could be combined to produce thè 
meaning of thè primitive. 

Means of Abstraction. In addition to primitives and means of combination, 
powerful languages have an additional type of component that enables eco- 
nomie communication: means of abstraction. 

Means of abstraction allow us to give a simple name to a complex entity. In 
English, thè means of abstraction are pronouns like “she”, “it”, and “they”. The 
meaning of a pronoun depends on thè context in which it is used. It abstraets 
a complex meaning with a simple word. For example, thè it in thè previous 
sentence abstraets “thè meaning of a pronoun”, but thè it in thè sentence before 
that one abstraets “a pronoun”. 

In naturai languages, there are a limited number of means of abstraction. En- 
glish, in particular, has a very limited set of pronouns for abstracting people. 
It has she and he for abstracting a female or male person, respectively, but no 
gender- neutral pronouns for abstracting a person of either sex. The interpre- 
tation of what a pronoun abstract in naturai languages is often confusing. For 
example, it is unclear what thè it in this sentence refers to. Languages for pro- 
gramming computers need means of abstraction that are both powerful and un- 
ambiguous. 


Exercise 2.1. According to thè Guinness Book of World Records, thè longest word 
in thè English language is floccinaucinihilipilification, meaning “The act or 
habit of describing or regarding something as worthless”. This word was reput- 
edly invented by a non-hippopotomonstrosesquipedaliophobic student at Eton 
who combined four words in his Latin textbook. Prove Guinness wrong by iden- 
tifying a longer English word. An English speaker (familiar with floccinaucinihil- 
ipilifìcation and thè morphemes you use) should be able to deduce thè meaning 
of your word. 

Exercise 2.2. Merriam-Webster’s word for thè year for 2006 was truthiness, a 
word invented and popularized by Stephen Colbert. Its defìnition is, “truth that 
Comes from thè gut, not books”. Identify thè morphemes that are used to build 
truthiness, and explain, based on its composition, what truthiness should mean. 

1 Guessing that it is a verb meaning to pass from thè solid to liquid state would also be reasonable. 
This shows how imprecise and ambiguous naturai languages are; for programming computers, we 
need thè meanings of constructs to be clearly determined. 
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Dictionaries are but 
thè depositories of 
words already 
legitimated by 
usage. Society is thè 
workshop in which 
new ones are 
elaborated. When 
an individuai uses a 
new word, ifill 
formed, it is 
rejected; ifwell 
formed, adopted, 
and after due time, 
laid up in thè 
depository of 
dictionaries. 

Thomas Jefferson, 
letter to John Adams, 
1820 

recursive transition 
network 


Exercise 2.3. According to thè Oxford English Dictionary, Thomas Jefferson is 
thè fìrst person to use more than 60 words in thè dictionary. Jeffersonian words 
include: (a) authentication, (b) belittle, (c) indecipherable, (d) inheritability, 
(e) odometer, (f) sanction, (g) vomit-grass, and (h) shag. For each Jeffersonian 
word, guess its derivation and explain whether or not its meaning could be in- 
ferred from its components. 

Exercise 2.4. Embiggening your vocabulary with anticromulent words ecdysi- 
asts can grok. 

a. Invent a new English word by combining common morphemes. 

b. Get someone else to use thè word you invented. 

c. [**] Convince Merriam-Webster to add your word to their dictionary. 

2.3 Recursive Transition Networks 

This section describes a more powerful technique for defining languages. The 
surface forms of a textual language are a (typically infinite) set of strings. To 
define a language, we need to define a System that produces all strings in thè 
language and no other strings. (The problem of associating meanings with those 
strings is more difficult; we consider it in later chapters.) 

A recursive transition network (RTN) is defined by a graph of nodes and edges. 
The edges are labeled with output symbols — these are thè primitives in thè lan- 
guage. The nodes and edge structure provides thè means of combination. 

One of thè nodes is designated thè start node (indicated by an arrow pointing 
into that node). One or more of thè nodes may be designated as final nodes 
(indicated by an inner circle). A string is in thè language if there exists some 
path from thè start node to a final node in thè graph where thè output symbols 
along thè path edges produce thè string. 

Figure 2.1 shows a simple RTN with three nodes and four edges that can produce 
four different sentences. Starting at thè node marked Noun, there are two possi- 
bile edges to follow. Each edge outputs a different Symbol, and leads to thè node 
marked Verb. From that node there are two output edges, each leading to thè fi- 
nal node marked S. Since there are no edges out of S, this ends thè string. Jfence, 
thè RTN can produce four strings corresponding to thè four different paths from 
thè start to final node: “Alice jumps’’, “Alice runs”, “Bob jumps”, and “Bob runs”. 

Recursive transition networks are more efficient than listing thè strings in a lan- 
guage, since thè number of possible strings increases with thè number of possi- 
bile paths through thè graph. For example, adding one more edge from Noun to 



Bob runs 

Figure 2.1. Simple recursive transition network. 
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Verb with label “Colleen” adds two new strings to thè language. 

The expressive power of recursive transition networks increases dramatically 
once we add edges that form cycles in thè graph. This is where thè recursive 
in thè name comes from. Once a graph has a cycle, there are inflnitely many 
possible paths through thè graph! 

Consider what happens when we add thè single “and” edge to thè previous net- 
work to produce thè network shown in Figure 2.2 below. 


and 



Bob runs 

Figure 2.2. RTN with a cycle. 


Now, we can produce infinitely many different strings! We can follow thè “and” 
edge back to thè Noun node to produce strings like “Alice runs and Bob jumps 
and Alice jumps” with as many conjuncts as we want. 


Exercise 2.5. Draw a recursive transition network that defìnes thè language of 
thè whole numbers: 0, 1, 2, . . . 

Exercise 2.6. How many different strings can be produced by thè RTN below: 


jumps 



Exercise 2.7. Recursive transition networks. 

a. How many nodes are needed for a recursive transition network that can pro- 
duce exactly 8 strings? 

b. How many edges are needed for a recursive transition network that can pro- 
duce exactly 8 strings? 

c. [*★] Given a whole number n, how many edges are needed for a recursive 
transition network that can produce exactly n strings? 

Subnetworks. In thè RTNs we have seen so far, thè labels on thè output edges 
are direct outputs known as terminals : following an edge just produces thè Sym- 
bol on that edge. We can make more expressive RTNs by allowing edge labels to 
also name subnetworks. A subnetwork is identifìed by thè name of its starting 
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node. When an edge labeled with a subnetwork is followed, thè network traver- 
sai jumps to thè subnetwork node. Then, it can follow any path from that node 
to a final node. Upon reaching a final node, thè network traversai jumps back to 
complete thè edge. 

For example, consider thè network shown in Figure 2.3. It describes thè same 
language as thè RTN in Figure 2.1, but uses subnetworks for Noun and Verb. To 
produce a string, we start in thè Sentence node. The only edge out from Sen- 
tence is labeled Noun. To follow thè edge, we jump to thè Noun node, which is 
a separate subnetwork. Now, we can follow any path from Noun to a final node 
(in this cases, outputting either “Alice” or “Bob” on thè path toward EndNoun. 



Figure 2.3. Recursive transition network with subnetworks. 


Suppose we replace thè Noun subnetwork with thè more interesting version 
shown in Figure 2.4.This subnetwork includes an edge from Noun to NI labeled 
Noun. Following this edge involves following a path through thè Noun subnet- 
work. Starting from Noun, we can generate complex phrases like “Alice and Bob” 
or “Alice and Bob and Alice” (fìnd thè two different paths through thè network 
that generate this phrase) . 



Figure 2.4. Alternate Noun subnetwork. 


To keep track of paths through RTNs without subnetworks, a single marker suf- 
fìces. We can start with thè marker on thè start node, and move it along thè path 
through each node to thè final node. Keeping track of paths on an RTN with 
subnetworks is more complicated. We need to keep track of where we are in thè 
current network, and also where to continue to when a final node of thè current 
subnetwork is reached. Since we can enter subnetworks within subnetworks, 
we need a way to keep track of arbitrarily many jump points. 

stack A stack is a useful way to keep track of thè subnetworks. We can think of a stack 
like a stack of trays in a cafeteria. At any point in time, only thè top tray on thè 
stack can be reached. We can pop thè top tray off thè stack, after which thè next 
tray is now on top. We can push a new tray on top of thè stack, which makes thè 
old top of thè stack now one below thè top. 

We use a stack of nodes to keep track of thè subnetworks as they are entered. 
The top of thè stack represents thè next node to process. At each step, we pop 
thè node off thè stack and follow a transition from that node. 
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EndNoun 


Stack 





N = Sentence 


N = Noun 


N = EndNoun 

Step 

1 

2 

3, 4, 5, 6 

2 

3, 4, 5, 6 

2, 3 

Output 





Alice 





III Verb 

Stack 


M EndSentence 

Step 

N- SI 

2 

3, 4, 5, 6 

Output 




N = Verb 

2 


Li i ti Ver b 

EndSentence I EndSentence 


N = EndVerb N = EndSentence 

3, 4, 5, 6 2, 3 2 

runs 


Figure 2.5. RTN generating “Alice runs”. 

Using a stack, we can derive a path through an RTN using this procedure: 

1. Initially, push thè starting node on thè stack. 

2. If thè stack is empty, stop. Otherwise, pop a node, N, off thè stack. 

3. If thè popped node, N, is a final node return to step 2. 2 

4. Select an edge from thè RTN that starts from node N. Use D to denote thè 
destination of that edge, and s to denote thè output Symbol on thè edge. 

5. Push D on thè stack. 

6. If s is a subnetwork, push thè node s on thè stack. Otherwise, output s, 
which is a terminal. 

7. Go back to step 2. 

Consider generating thè string “Alice runs” using thè RTN in Figure 2.3. We start 
following step 1 by pushing Sentence on thè stack. In step 2, we pop thè stack, 
so thè current node, N, is Sentence. Since Sentence is not a final node, we do 
nothing for step 3. In step 4, we follow an edge starting from Sentence. There is 
only one edge to choose and it leads to thè node labeled SI. In step 5, we push 
SI on thè stack. The edge we followed is labeled with thè node Noun, so we 
push Noun on thè stack. The stack now contains two items: [Noun, SI]. Since 
Noun is on top, this means we will frrst traverse thè Noun subnetwork, and then 
continue from SI. 

As directed by step 7, we go back to step 2 and continue by popping thè top 
node, Noun, off thè stack. It is not a final node, so we continue to step 4, and 
select thè edge labeled “Alice” from Noun to EndNoun. We push EndNoun on 
thè stack, which now contains: [EndNoun, SI]. The label on thè edge is thè ter- 
minal, “Alice”, so we output “Alice” following step 6. We continue in thè same 
manner, following thè steps in thè procedure as we keep track of a path through 
thè network. The full processing steps are shown in Figure 2.5. 


Exercise 2.8. Show thè sequence of stacks used in generating thè string “Alice 
and Bob and Alice runs" using thè network in Figure 2.3 with thè alternate Noun 
subnetwork from Figure 2.4. 

2 For simplicity, this procedure assumes we always stop when a final node is reached. RTNs can 
have edges out of final nodes (as in Figure 2.2) where it is possible to either stop or continue from a 
final node. 
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grommar 



John Backus 


Iflunked out every 
year. I never 
studied. I hated 
studying. I wasjust 
gooflng around. It 
had thè delightful 
consequence that 
every year I went to 
summer school in 
New Hampshire 
where I spent thè 
summer sailing and 
having anice time. 

John Backus 


Exercise 2.9. Identify a string that cannot be produced using thè RTN from 
Figure 2.3 with thè alternate Noun subnetwork from Figure 2.4 without thè stack 
growing to contain five elements. 

Exercise 2.10. The procedure given for traversing RTNs assumes that a subnet- 
work path always stops when a final node is reached. Hence, it cannot follow all 
possible paths for an RTN where there are edges out of a final node. Describe a 
procedure that can follow all possible paths, even for RTNs that include edges 
from final nodes. 

2.4 Replacement Grammars 

Another way to define a language is to use a grammar. This is thè most common 
way languages are defined by computer scientists today, and thè way we will use 
for thè rest of this hook. 

A grammar is a set of rules for generating all strings in thè language. We use 
thè Backus-Naur Form (BNF) notation to define a grammar. BNF grammars are 
exactly as powerful as recursive transition networks (Exploration 2.1 explains 
what this means and why it is thè case), but easier to write down. 

BNF was invented by John Backus in thè late 1950s. Backus led efforts at IBM 
to define and implement Fortran, thè first widely used programming language. 
Fortran enabled computer programs to be written in a language more like famil- 
iar algebraic formulas than low-level machine instructions, enabling programs 
to be written more quickly. In defining thè Fortran language, Backus and his 
team used ad hoc English descriptions to define thè language. These ad hoc 
descriptions were often misinterpreted, motivating thè need for a more precise 
way of defining a language. 

Rules in a Backus-Naur Form grammar have thè form: 

nonterminal ::=^> replacement 

The left side of a rule is always a single Symbol, known as a nonterminal since it 
can never appear in thè final generated string. The right side of a rule contains 
one or more symbols. These symbols may include nonterminals, which will be 
replaced using replacement rules before generating thè final string. They may 
also be terminals, which are output symbols that never appear as thè left side 
of a rule. When we describe grammars, we use italics to represent nonterminal 
symbols, and bold to represent terminal symbols. The terminals are thè primi- 
tives in thè language; thè grammar rules are its means of combination. 

We can generate a string in thè language described by a replacement grammar 
by starting from a designated start Symbol (e.g., sentence ), and at each step se- 
lecting a nonterminal in thè working string, and replacing it with thè right side of 
a replacement rule whose left side matches thè nonterminal. Wherever we find 
a nonterminal on thè left side of a rule, we can replace it with what appears on 
thè right side of any rule where that nonterminal matches thè left side. A string 
is generated once there are no nonterminals remaining. 

Here is an example BNF grammar (that describes thè same language as thè RTN 
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in Figure 2.1): 


1 . 

Sentence 

:=> Noun Verb 

2. 

Noun 

:=> Alice 

3. 

Noun 

:=> Bob 

4. 

Verb 

:=>• jumps 

5. 

Verb 

:=>• runs 


Starting from Sentence, thè grammar can generate four sentences: “Alice jumps”, 
“Alice runs’’, “Bob jumps”, and “Bob runs”. 

A derivation shows how a grammar generates a given string. Here is thè deriva- 
tion of “Alice runs”: 

Sentence ::=> Noun Verb using Rule 1 

::=> Alice Verb replacing Nauti using Rule 2 

::=> Alice runs replacing Verb using Rule 5 


We can represent a grammar derivation as a tree, where thè root of thè tree is 
thè starting nonterminal ( Sentence in this case), and thè leaves of thè tree are 
thè terminals that form thè derived sentence. Such a tree is known as a parse 
tree. Here is thè parse tree for thè derivation of “Alice runs”: 


Sentence 



Noun 

Verb 

Alice 

runs 


BNF grammars can be more compact than just listing strings in thè language 
since a grammar can have many replacements for each nonterminal. For exam- 
ple, adding thè rule, Noun ::=> Colleen, to thè grammar adds two new strings 
(“Colleen runs” and “Colleen jumps”) to thè language. 

Recursive Grammars. The reai power of BNF as a compact notation for describ- 
ing languages, though, comes once we start adding recursive rules to our gram- 
mar. A grammar is recursive if thè grammar contains a nonterminal that can 
produce a production that contains itself. 

Suppose we add thè rule, 

Sentence ::=> Sentence and Sentence 

to our example grammar. Now, how many sentences can we generate? 

Infrnitely many! This grammar describes thè same language as thè RTN in Fig- 
ure 2.2. It can generate “Alice runs and Bob jumps” and “Alice runs and Bob 
jumps and Alice runs” and sentences with any number of repetitions of “Alice 
runs”. This is very powerful: by using recursive rules a compact grammar can be 
used to define a language containing infrnitely many strings. 


derivation 


parse tree 
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1 Example 2.1: Whole Numbers j 

This grammar dehnes thè language of thè whole numbers (0, 1, . . .) with leading 

zeros allowed: 



Number 

:=> Digit MoreDigits 


MoreDigits 

: => 

Digit ::=> 4 

MoreDigits 

:=> Number 

Digit ::=> 5 

Digit 

:=>- 0 

Digit ::=> 6 

Digit : 

:=>- 1 

Digit ::=> 7 

Digit : 

:=>- 2 

Digit: :=> 8 

Digit : 

:=>- 3 

Digit ::=> 9 


Here is thè parse tree for a derivation of 37 from Number. 

Number 

Digit 

I 

3 

Digit MoreDigits 

I I 

7 e 

Circular vs. Recursive Definitions. The second rute means we can replace 
MoreDigits with nothing. This is sometimes written as e to make it clear that thè 
replacement is empty: MoreDigits ::=> e. 

This is a very important rule in thè grammar — without it no strings could be gen- 
erated; with it infinitely many strings can be generated. The key is that we can 
only produce a string when all nonterminals in thè string have been replaced 
with terminals. Without thè MoreDigits e rule, thè only rule we would have 
with MoreDigits on thè left side is thè third rule: MoreDigits ::=> Number. 

The only rule we have with Number on thè left side is thè hrst rule, which re- 
places Number with Digit MoreDigits. Every time we follow this rule, we replace 
MoreDigits with Digit MoreDigits. We can produce as many Digits as we want, 
but without thè MoreDigits ::=> e rule we can never stop. 

This is thè difference between a circular dehnition, and a recursive dehnition. 
Without thè stopping rule, MoreDigits would be defìned in a circular way. There 
is no way to start with MoreDigits and generate a production that does not con- 
tain MoreDigits (or a nonterminal that eventually must produce MoreDigits ). 
With thè MoreDigits e rule, however, we have a way to produce something 
base case terminal from MoreDigits. This is known as a base case — a rule that turns an 
otherwise circular dehnition into a meaningful, recursive dehnition. 

Condensed Notation. It is common to have many grammar rules with thè same 
left side nonterminal. For example, thè whole numbers grammar has ten rules 
with Digit on thè left side to produce thè ten terminal digits. Each of these is an 
alternative rule that can be used when thè production string contains thè non- 
terminal Digit. A compact notation for these types of rules is to use thè vertical 


MoreDigits 

Number 
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bar (|) to separate alternative replacements. For example, we could write thè ten 
Digit rules compactly as: 

Digit 0|1|2|3|4|5|6|7|8|9 


Exercise2.11. Suppose we replaced thè frrst rule ( Number ::=> Digit MoreDigits) 
in thè whole numbers grammar with: Number ::=>• MoreDigits Digit. 

a. How does this change thè parse tree for thè derivation of 37? Draw thè parse 
tree that results from thè new grammar. 

b. Does this change thè language? Either show some string that is in thè lan- 
guage defrned by thè modified grammar but not in thè originai language (or 
vice versa), or argue that both grammars generate thè same strings. 


Exercise 2.12. The grammar for whole numbers we defrned allows strings with 
non-standard leading zeros such as “000” and “00005”. Devise a grammar that 
produces all whole numbers (including “0”), but no strings with unnecessary 
leading zeros. 


Exercise 2.13. Defìne a BNF grammar that describes thè language of decimai 
numbers (thè language should include 3.14159, 0.423, and 1120 but not 1.2.3). 


Exercise 2.14. The BNF grammar below (extracted from Paul Mockapetris, Do- 
main Names - Implementation and Specification, IETF RFC 1035) describes thè 
language of domain names on thè Internet. 


Domain 

SubDomainList 

Label 

MoreLetters 

LetterHyphens 

LDHyphen 

LetterDigit 

Letter 

Digit 


SubDomainList 

Label \ SubDomainList . Label 

Letter MoreLetters 

LetterHyphens LetterDigit \ e 

LDHyphen \ LDHyphen LetterHyphens \ e 

LetterDigit \ - 

Letter \ Digit 

A|B| ... |Z|a|b| ... |z 

0|1|2|3|4|5|6|7|8|9 


a. Show a derivation for www.virginia.edu in thè grammar. 

b. According to thè grammar, which of thè following are valid domain names: 

(1) tj, (2) a.-b.c, (3) a-a.b-b.c-c, (4) a.g.r.e.a.t.d.o.m.a.i.n-. 


Exploration 2.1: Power of Language Systems 


Section 2.4 claimed that recursive transition networks and BNF grammars are 
equally powerful. What does it mean to say two systems are equally powerful? 

A language description mechanism is used to defìne a set of strings comprising a 
language. Hence, thè power of a language description mechanism is determined 
by thè set of languages it can defìne. 

One approach to measure thè power of language description mechanism would 
be to count thè number of languages that it can defìne. Even thè simplest mech- 
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anisms can define infinitely many languages, however, so just counting thè num- 
ber of languages does not distinguish well between thè different language de- 
scription mechanisms. Both RTNs and BNFs can describe infinitely many differ- 
ent languages. We can always add a new edge to an RTN to increase thè number 
of strings in thè language, or add a new replacement rule to a BNF that replaces 
a nonterminal with a new terminal symbol. 

Instead, we need to consider thè set of languages that each mechanism can de- 
fine. A System A is more powerful that another System B if we can use A to 
define every language that can be defined by B, and there is some language L 
that can be defined using A that cannot be defined using B. This matches our 
intuitive interpretation of more powerful — A is more powerful than B if it can 
do everything B can do and more. 

The diagrams in Figure 2.6 show three possible scenarios. In thè leftmost pie- 
ture, thè set of languages that can be defined by B is a proper subset of thè set 
of languages that can be defined by A. Hence, A is more powerful than B. In 
thè center picture, thè sets are equal. This means every language that can be de- 
fined by A can also be defined by B, and every language that can be defined by B 
can also be defined by A, and thè Systems are equally powerful. In thè rightmost 
picture, there are some elements of A that are not elements of B, but there are 
also some elements of B that are not elements of A. This means we cannot say 
either one is more powerful; A can do some things B cannot do, and B can do 
some things A cannot do. 



A is more powerful than B 



A is as powerful as B 



A and B are not comparatole 


Figure 2.6. System power relationships. 


To determine thè relationship between RTNs and BNFs we need to understand 
if there are languages that can be defined by a BNF that cannot be defined by 
a RTN and if there are languages that can be defined by a RTN that cannot be 
defined by an BNF. We will show only thè first part of thè proof here, and leave 
thè second part as an exercise. 

For thè first part, we prove that there are no languages that can be defined by a 
BNF that cannot be defined by an RTN. Equivalently, every language that can be 
defined by a BNF grammar has a corresponding RTN. Since there are infinitely 
many languages that can be defined by BNF grammars, we cannot prove this 
by enumerating each language and showing its corresponding RTN. Instead, we 
use a proof technique commonly used in computer Science: proof by construc- 
proofby tion. We show an algorithm that given any BNF grammar constructs an RTN 
construction that defines thè same language as thè input BNF grammar. 

Our strategy is to construct a subnetwork corresponding to each nonterminal. 
For each rule where thè nonterminal is on thè left side, thè right hand side is 
converted to a path through that node’s subnetwork. 
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Before presenting thè generai construction algorithm, we illustrate thè approach 
with thè example BNF grammar from Example 2.1: 

Number ::=> Digit MoreDigits 
MoreDigits ::=> e 
MoreDigits ::=> Number 
Digit 0|1|2|3|4|5|6|7|8|9 


The grammar has three nonterminals: Number, Digit, and MoreDigits. For each 
nonterminal, we construct a sub network by fìrst creating two nodes correspond- 
ing to thè start and end of thè subnetwork for thè nonterminal. We make Start- 
Number thè start node for thè RTN since Number is thè starting nonterminal for 
thè grammar. 

Next, we need to add edges to thè RTN corresponding to thè production rules 
in thè grammar. The fìrst rule indicates that Number can be replaced by Digit 
MoreDigits. To make thè corresponding RTN, we need to introduce an interme- 
diate node since each RTN edge can only contain one label. We need to traverse 
two edges, with labels StartDigit and StartMoreDigits between thè StartNumber 
and EndNumber nodes. The resulting partial RTN is shown in Figure 2.7. 


tartMoreDigits 

“ A 



Figure 2.7. Converting thè Number productions to an RTN. 

For thè MoreDigits nonterminal there are two productions. The fìrst means 
MoreDigits can be replaced with nothing. In an RTN, we cannot have edges 
with unlabeled outputs. So, thè equivalent of outputting nothing is to turn Start- 
MoreDigits into a final node. The second production replaces MoreDigits with 
Number. We do this in thè RTN by adding an edge between StartMoreDigits and 
EndMoreDigits labeled with Number, as shown in Figure 2.8. 



Figure 2.8. Converting thè MoreDigits productions to an RTN. 


Finally, we convert thè ten Digit productions. For each rule, we add an edge 
between StartDigit and EndDigit labeled with thè digit terminal, as shown in 
Figure 2.9. 

This example illustrates that it is possible to convert a particular grammar to an 
RTN. For a generai proof, we present a generai an algorithm that can be used to 
do thè same conversion for any BNF: 


1. For each nonterminal X in thè grammar, construct two nodes, StartX and 
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Figure 2.9. Converting thè Digit productions to an RTN. 

EndX, where EndX is a final node. Make thè node StartS thè start node of 
thè RTN, where S is thè start nonterminal of thè grammar. 

2. For each rute in thè grammar, add a corresponding path through thè RTN. 
All BNF rules have thè form X ::=> replacement where X is a nonterminal 
in thè grammar and replacement is a sequence of zero or more terminals 
and nonterminals: [Ro, Ri, . . . , R n ] . 

(a) If thè replacement is empty, make StartX a final node. 

(b) If thè replacement has just one element, Ro, add an edge from StartX 
to EndX with edge label Ro- 

(c) Otherwise: 

i. Add an edge from StartX to a new node labeled X, 0 (where i iden- 
tifies thè grammar rule), with edge label R 0 . 

ii. For each remaining element Rj in thè replacement add an edge 
from Xi j _ i to a new node labeled X hj with edge label R ; . (For 
example, for element R 1; a new node X, j is added, and an edge 
from X ir0 to X, i with edge label Ri .) 

iii. Add an edge from X; /W _i to EndX with edge label R„. 

Following this procedure, we can convert any BNF grammar into an RTN that 
defines thè same language. Hence, we have proved that RTNs are at least as 
powerful as BNF grammars. 

To complete thè proof that BNF grammars and RTNs are equally powerful ways 
of defining languages, we also need to show that a BNF can define every lan- 
guage that can be defined using an RTN. This part of thè proof can be done 
using a similar strategy in reverse: by showing a procedure that can be used to 
construct a BNF equivalent to any input RTN. We leave thè details as an exercise 
for especially ambitious readers. 


Exercise 2.15. Produce an RTN that defines thè same languages as thè BNF 
grammar from Exercise 2.14. 

Exercise 2. 16. [*] Prove that BNF grammars are as powerful as RTNs by devising 
a procedure that can construct a BNF grammar that defines thè same language 
as any input RTN. 

2.5 Summary 

Languages define a set of surface forms and associated meanings. Since use- 
ful language must be able to express infinitely many things, we need tools for 
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defining infinite sets of surface forms using compact and precise notations. The 
tool we will use for thè remainder of this hook is thè BNF replacement gram- 
mar which precisely defines a language using replacement rules. This System 
can describe infinite languages with small representations because of thè power 
of recursive rules. In thè next chapter, we introduce thè Scheme programming 
language that we will use to describe procedures. 
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online library 


3 

Programming 


The Analytical Engine has no pretensions whatever to originate any thing. It cari 
do whatever we know how to order it to perfonn. It canfollow analysis; but it 
has no power of anticipating any analytical relations or truths. Its province is to 
assist us in making available what we are already acquainted with. 

Augusta Ada Countess of Lovelace, in Notes on thè Analytical Engine, 1843 


What distinguishes a computer from other machines is its programmability. 
Without a program, a computer is an overpriced door stopper. With thè right 
program, though, a computer can be a tool for communicating across thè conti- 
nent, discovering a new molecule that can cure cancer, composing a symphony, 
or managing thè logistics of a retail empire. 

Programming is thè act ofwriting instructions that make thè computer do some- 
thing useful. It is an intensely creative activity, involving aspects of art, engi- 
neering, and Science. Good programs are written to be executed efficiently by 
computers, but also to be read and understood by humans. The best programs 
are delightful in ways similar to thè best architecture, elegant in both form and 
function. 



Golden Gate Bridge 

The ideal programmer would have thè Vision of Isaac Newton, thè mtellect of 
Albert Einstein, thè creativity of Miles Davis, thè aesthetic sense of Maya Lin, 
thè wisdom of Benjamin Franklin, thè literary talent of William Shakespeare, 
thè oratorical skills of Martin Luther King, thè audacity of John Roebling, and 
thè self-confidence of Grace Hopper. 


Fortunately, it is not necessary to possess all of those rare qualities to be a good 
programmer! Indeed, anyone who is able to master thè intellectual challenge 
of learning a language (which, presumably, anyone who has gotten this far has 
done at least for English) can become a good programmer. Since programming 
is a new way of thinking, many people hnd it challenging and even frustrating 
at hrst. Because thè computer does exactly what it is told, a small mistake in a 
program may prevent it from working as intended. With a bit of patience and 
persistence, however, thè tedious parts of programming become easier, and you 
will be able to focus your energies on thè fun and creative problem solving parts. 


In thè previous chapter, we explored thè components of language and mecha- 
nisms for defming languages. In this chapter, we explain why naturai languages 
are not a satisfactory way for defming procedures and introduce a language for 
programming computers and how it can be used to dehne procedures. 
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3. 1 Problems with Naturai Languages 

Naturai languages, such as English, work adequately (most, but certainly not 
all, of thè time) for human-human communication, but are not well-suited for 
human-computer or computer-computer communication. Why can’t we use 
naturai languages to program computers? 

Next, we survey several of thè reasons for this. We use specifrcs from English, 
although all naturai languages suffer from these problems to varying degrees. 


Complexity. Although English may seem simple to you now, it took many years 
of intense effort (most of it subconscious) for you to learn it. Despite using it for 
most of their waking hours for many years, native English speakers know a small 
fraction of thè entire language. The Oxford English Dictionary contains 615,000 
words, of which a typical native English speaker knows about 40,000. 

Ambiguity. Not only do naturai languages have huge numbers of words, most 
words have many different meanings. Understanding thè intended meaning of 
an utterance requires knowing thè context, and sometimes pure guesswork. 

For example, what does it mean to be paid biweeklyl According to thè American 
Heritage Dictionary 1 , biweekly has two definitions: 

1. Happening every two weeks. 

2. Happening twice a week; semiweekly. 

Merriam-Webster’s Dictionary 2 takes thè opposite approach: 

1. occurring twice a week 

2. occurring every two weeks : fortnightly 

So, depending on which defmition is intended, someone who is paid biweekly 
could either be paid once or four times every two weeks! The behavior of a pay- 
roll management program better not depend on how biweekly is interpreted. 

Even if we can agree on thè defmition of every word, thè meaning of a sentence 
is often ambiguous. This particularly difficult example is taken from thè instruc- 
tions with a shipment of ballistic missiles from thè British Admiralty: 3 

It is necessaryfor technical reasons that these warheads be stored upside 
down, that is, with thè top at thè bottoni and thè bottom at thè top. In 
order that there be no doubt as to which is thè bottom and which is thè 
top, for Storage purposes, it will be seen that thè bottom ofeach warhead 
has been labeled ’TOP’. 

Irregularity. Because naturai languages evolve over time as different cultures 
internet and speakers misspeak and listeners mishear, naturai languages end up 
a morass of irregularity. Nearly all grammar rules have exceptions. For example, 
English has a rule that we can make a word plural by appending an s. The new 

1 American Heritage, Dictionaiy of thè English Language (Fourth Edition) , Houghton Mifflin Com- 
pany, 2007 (http://www.answers.com/biweekly). 

2 Merriam-Webster Online, Merriam-Webster, 2008 (http://www.merriam-webster.com/dictionary/ 
biweekly). 

3 Carl C. Gaither and Alma E. Cavazos-Gaither, Practically Speaking: A Dictionaiy ofQuotations 
on Engineering, Technology and Architecture, Taylor & Francis, 1998. 
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word means “more than one of thè originai word’s meaning”. This rule works for 
most words: word words, language i— s- languages, persoti i— s- persons . 4 

It does not work for all words, however. The plural of goose is geese (and gooses 
is not an English word), thè plural of deer is deer (and deers is not an English 
word), and thè plural of beer is controversial (and may depend on whether you 
speak American English or Canadian English) . 

These irregularities can be charming for a naturai language, but they are a Con- 
stant source of difficulty for non-native speakers attempting to learn a language. 
There is no sure way to predict when thè rule can be applied, and it is necessary 
to memorize each of thè irregular forms. 

Uneconomic. It requires a lot of space to express a complex idea in a naturai lan- 
guage. Many superfluous words are needed for grammatical correctness, even 
though they do not contribute to thè desired meaning. Since naturai languages 
evolved for everyday communication, they are not well suited to describing thè 
precise steps and decisions needed in a computer program. 

As an example, consider a procedure for hnding thè maximum of two numbers. 
In English, we could describe it like this: 

Toflnd thè maximum oftwo numbers, compare them. Ifthefirst num- 
ber is greater than thè second number, thè maximum is thefirst number. 
Otherwise, thè maximum is thè second number. 

Perhaps shorter descriptions are possible, but any much shorter description 
probably assumes thè reader already knows a lot. By contrast, we can express 
thè same steps in thè Scheme programming language in very concise way (don’t 
worry if this doesn’t make sense yet — it should by thè end of this chapter): 

(define ( bigger a b) (if (> ab) a b)) 

Limited means of abstraction. Naturai languages provide small, fixed sets of 
pronouns to use as means of abstraction, and thè rules for binding pronouns to 
meanings are often unclear. Since programming often involves using simple 
names to refer to complex things, we need more powerful means of abstraction 
than naturai languages provide. 


3.2 Programming Languages 

For programming computers, we want simple, unambiguous, regular, and eco- 
nomical languages with powerful means of abstraction. A programming lan- 
guage is a language that is designed to be read and written by humans to create 
programs that can be executed by computers. 

Programming languages come in many flavors. It is difficult to simultaneously 
satisfy all desired properties since simplicity is often at odds with economy. Ev- 
ery feature that is added to a language to increase its expressiveness incurs a cost 
in reducing simplicity and regularity. For thè first two parts of this hook, we use 
thè Scheme programming language which was designed primarily for simplic- 
ity. For thè later parts of thè hook, we use thè Python programming language, 
which provides more expressiveness but at thè cost of some added complexity. 


I have ma.de this 
letter longer than 
usuai, only because 
I have nothad thè 
time to make it 
shorter. 

Blaise Pascal, 1657 


programming 

language 


4 Or is it peoplel What is thè singular of peoplel What about peepst Can you only have one peept 
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Grace Hopper 

Image courtesy Computer 
History Museum (1952) 


compiler 


interpreter 


Nobody believed 
thatlhada 
running compiler 
and nobody would 
touch it. They told 
me computers could 
only do arìthmetic. 
Grace Hopper 


Another reason there are many different programming languages is that they 
are at different levels of abstraction. Some languages provide programmers with 
detailed control over machine resources, such as selecting a particular location 
in memory where a value is stored. Other languages hide most of thè details of 
thè machine operation from thè programmer, allowing them to focus on higher- 
level actions. 

Ultimately, we want a program thè computer can execute. This means at thè 
lowest level we need languages thè computer can understand directly. At this 
level, thè program is just a sequence of bits encoding machine instructions. 
Code at this level is not easy for humans to understand or write, but it is easy 
for a processor to execute quickly. The machine code encodes instructions that 
direct thè processor to take simple actions like moving data from one place to 
another, performing simple arìthmetic, and jumping around to find thè next in- 
struction to execute. 

For example, thè bit sequence 1110101111111110 encodes an instruction in thè 
Intel x86 instruction set (used on most PCs) that instructs thè processor to jump 
backwards two locations. Since thè instruction itself requires two locations of 
space, jumping back two locations actually jumps back to thè beginning of this 
instruction. Hence, thè processor gets stuck running forever without making 
any progress. 

The computer’s processor is designed to execute very simple instructions like 
jumping, adding two small numbers, or comparing two values. This means each 
instruction can be executed very quickly. A typical modem processor can exe- 
cute billions of instructions in a second. 5 

Until thè early 1950s, all programming was done at thè level of simple instruc- 
tions. The problem with instructions at this level is that they are not easy for 
humans to write and understand, and you need many simple instructions be- 
fore you have a useful program. 

A compiler is a computer program that generates other programs. It translates 
an input program written in a high-level language that is easier for humans to 
create into a program in a machine-level language that can be executed by thè 
computer. Admiral Grace Hopper developed thè first compilers in thè 1950s. 

An alternative to a compiler is an interpreter. An interpreter is a tool that trans- 
lates between a higher-level language and a lower-level language, but where a 
compiler translates an entire program at once and produces a machine language 
program that can be executed directly, an interpreter interprets thè program a 
small piece at a time while it is running. This has thè advantage that we do not 
have to run a separate tool to compile a program before running it; we can sim- 
ply enter our program into thè interpreter and run it right away. This makes it 
easy to make small changes to a program and try it again, and to observe thè 
state of our program as it is running. 

One disadvantage of using an interpreter instead of a compiler is that because 
thè translation is happening while thè program is running, thè program exe- 
cutes slower than a compiled program. Another advantage of compilers over 

5 A "2GHz processor” executes 2 billion cycles per second. This does not map directly to thè num- 
ber of instructions it can execute in a second, though, since some instructions take several cycles to 
execute. 
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interpreters is that since thè compiler translates thè entire program it can also 
analyze thè program for consistency and detect certain types of programming 
mistakes automatically instead of encountering them when thè program is run- 
ning (or worse, not detecting them at all and producing unintended results). 
This is especially important when writing criticai programs such as flight con- 
trol software — we want to detect as many problems as possible in thè flight 
control software before thè piane is flying! 

Since we are more concerned with interactive explo radon than with performance 
and detecting errors early, we use an interpreter instead of a compiler. 


3.3 Scheme 

The programming System we use for thè fìrst part of this hook is depicted in 
Figure 3.1. The input to our programming System is a program written in a pro- 
gramming language named Scheme. A Scheme interpreter interprets a Scheme 
program and executes it on thè machine processor. 

Scheme was developed at MIT in thè 1970s by Guy Steele and Gerald Sussman, 
based on thè LISP programming language that was developed by John McCarthy 
in thè 1950s. Although many large Systems have been built using Scheme, it is 
not widely used in industry. It is, however, a great language for learning about 
computing and programming. The primary advantage of using Scheme to learn 
about computing is its simplicity and elegance. The language is simple enough 
that this chapter covers nearly thè entire language (we defer describing a few 
aspects until Chapter 9), and by thè end of this hook you will know enough to 
implement your own Scheme interpreter. By contrast, some programming lan- 
guages that are widely used in industriai programming such as C++ and Java 
require thousands of pages to describe, and even thè world’s experts in those 
languages do not agree on exactly what all programs mean. 


Scheme Program 


Interpreter 

(DrRacket) 


T 


Processor 


(define (bigger a b) 
(if (> a b) a b)) 

(bigger 3 4) 







1 0 'bugtf Ch«»5,™.0. a. r ® 

(define (bigger a b) 

(if (> a b) a b)) 

(bigger 3 4) 


Welcome to DrRacket. version 5.1.1 r3ml. 
Language: Pretty Big; memory limit: 128 MB. 

4 


» * 



Figure 3.1. Running a Scheme program. 
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Although almost everything we describe should work in all Scheme interpreters, 
for thè examples in this book we assume thè DrRacket programming environ- 
ment which is freely available from http://racket-lang.org/. DrRacket includes 
interpreters for many different languages, so you must select thè desired lan- 
guage using thè Language menu. The selected language defines thè grammar 
and evaluation rules that will be used to interpret your program. For all thè ex- 
amples in this book, we use a version of thè Scheme language named Pretty Big. 

3.4 Expressions 

A Scheme program is composed of expressions and definitions (we cover defi- 
expression nitions in Section 3.5). An expression is a syntactic element that has a value. 

The act of determining thè value associated with an expression is called evalua- 
evaluation tion. A Scheme interpreter, such as thè one provided in DrRacket, is a machine 
for evaluating Scheme expressions. If you enter an expression into a Scheme 
interpreter, thè interpreter evaluates thè expression and displays its value. 

Expressions may be primitives. Scheme also provides means of combination 
for producing complex expressions from simple expressions. The next subsec- 
tions describe primitive expressions and application expressions. Section 3.6 
describes expressions for making procedures and Section 3.7 describes expres- 
sions that can be used to make decisions. 

3.4.1 Primitives 

An expression can be replaced with a primitive: 

Expression ::=> PrìmitiveExpression 

As with naturai languages, primitives are thè smallest units of meaning. Hence, 
thè value of a primitive is its pre-defined meaning. 

Scheme provides many different primitives. Three useful types of primitives are 
described next: numbers, Booleans, and primitive procedures. 

Numbers. Numbers represent numerical values. Scheme provides all thè kinds 
of numbers you are familiar with including whole numbers, negative numbers, 
decimals, and rational numbers. 

Example numbers include: 

150 0 -12 

3.14159 3/4 999999999999999999999 

Numbers evaluate to their value. For example, thè value of thè primitive expres- 
sion 1 120 is 1120. 

Booleans. Booleans represent truth values. There are two primitives for repre- 
senting true and false: 

PrimitiveExpression ::=>- true \ false 

The meaning of true is true, and thè meaning of false is false. In thè DrRacket 
interpreter, #t and #f are used to represent thè primitive truth values. So, thè 
value true appears as #t in thè interactions window. 
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Symbol 

Description 

Inputs 

Output 

+ 

add 

zero or more 

numbers 

sum of thè input numbers (0 if 
there are no inputs) 

* 

multiply 

zero or more 

numbers 

product of thè input numbers (1 if 
there are no inputs) 

- 

subtract 

two numbers 

thè value of thè fìrst number minus 
thè value thè second number 

/ 

divide 

two numbers 

thè value of thè fìrst number 
divided by thè value of thè second 
number 

zero? 

is zero? 

one number 

true if thè input value is 0, 
otherwise false 

= 

is equal to? 

two numbers 

true if thè input values have thè 
same value, otherwise false 

< 

is less than? 

two numbers 

true if thè fìrst input value has 
lesser value than thè second input 
value, otherwise false 

> 

is greater than? 

two numbers 

true if thè fìrst input value has 
greater value than thè second input 
value, otherwise false 

<= 

is less than or 
equal to? 

two numbers 

true if thè fìrst input value is not 
greater than thè second input 
value, otherwise false 

>= 

is greater than or 
equal to? 

two numbers 

true if thè fìrst input value is not 
less than thè second input value, 
otherwise false 


Table 3.1. Selected Scheme Primitive Procedures. 

All of these primitive procedures operate on numbers. The fìrst four are thè basic arith- 
metic operators; thè rest are comparison procedures. Some of these procedures are 
defìned for more inputs than just thè ones shown here (e.g., thè subtract procedure also 
works on one number, producing its negation). 

Primitive Procedures. Scheme provides primitive procedures corresponding to 
many common functions. Mathematically, a function is a mapping from inputs function 
to outputs. For each valid input to thè function, there is exactly one associated 
output. For example, + is a procedure that takes zero or more inputs, each of 
which must be a number. Its output is thè sum of thè values of thè inputs. Table 
3.1 describes some primitive procedures for performing arithmetic and com- 
parisons on numbers. 

3.4.2 Application Expressions 

Most of thè actual work done by a Scheme program is done by application ex- 
pressions that apply procedures to operands. The expression (+ 1 2) is an Appli- 
cationExpression, consisting of three subexpressions. Although this example is 
probably simple enough that you can probably guess that it evaluates to 3, we 
will show in detail how it is evaluated by breaking down into its subexpressions 
using thè grammar rules. The same process will allow us to understand how any 
expression is evaluated. 
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3.4. Expressions 


The grammar rule for application is: 


Expression ::=^> ApplicationExpression 

ApplicationExpression ::=> ( Expression MoreExpressions) 
MoreExpressions ::=>- e \ Expression MoreExpressions 


This rule produces a list of one or more expressions surrounded by parentheses. 
The value of thè first expression should be a procedure; thè remaining expres- 
operands sions are thè inputs to thè procedure known as operands. Another name for 
arguments operands is arguments. 

Here is a parse tree for thè expression (+1 2): 


Expression 


Applica tionExpression 


Expression 


MoreExpressions 


PrìmitiveExpression 


Expression 


MoreExpressions 


+ 


PrìmitiveExpression Expression 


MoreExpressions 


Primi tiveExpression 


Following thè grammar rules, we replace Expression with ApplicationExpression 
at thè top of thè parse tree. Then, we replace ApplicationExpression with ( Ex- 
pression MoreExpressions ). The Expression term is replaced PrimitiveExpres- 
sion, and finally, thè primitive addition procedure +. This is thè first subexpres- 
sion of thè application, so it is thè procedure to be applied. The MoreExpres- 
sions term produces thè two operand expressions: 1 and 2, both of which are 
primitives that evaluate to their own values. The application expression is eval- 
uated by applying thè value of thè first expression (thè primitive procedure +) to 
thè inputs given by thè values of thè other expressions. Following thè meaning 
of thè primitive procedure, (+ 1 2) evaluates to 3 as expected. 

The Expression nonterminals in thè application expression can be replaced with 
anything that appears on thè right side of an expression rule, including an Ap- 
plicationExpression. 

We can build up complex expressions like (+(* 10 10) (+ 25 25)). Its parse tree 
is: 


Chapter3. Programming 


43 


Expression 


ApplicationExpression 



Expression 


MoreExpressions 



PrimitiveExpression 


Expression 


MoreExpressions 



+ 


ApplicationExpression Expression MoreExpressions 



(*1010) ApplicationExpression 


e 



(+ 25 25) 


This tree is similar to thè previous tree, except instead of thè subexpressions 
of thè first application expression being simple primitive expressions, they are 
now application expressions. (Instead of showing thè complete parse tree for 
thè nested application expressions, we use triangles.) 

To evaluate thè output application, we need to evaluate all thè subexpressions. 
The first subexpression, +, evaluates to thè primitive procedure. The second 
subexpression, (* 10 10), evaluates to 100, and thè third expression, (+ 25 25), 
evaluates to 50. Now, we can evaluate thè originai expression using thè values 
for its three component subexpressions: (+ 1 00 50) evaluates to 1 50. 

Exercise 3.1. Draw a parse tree for thè Scheme expression (+ 1 00 (* 5 (+ 5 5))) 
and show how it is evaluated. 

Exercise 3.2. Predict how each of thè following Scheme expressions is evalu- 
ated. After making your prediction, try evaluating thè expression in DrRacket. If 
thè result is different from your prediction, explain why thè Scheme interpreter 
evaluates thè expression as it does. 

a. 1120 

b. (+1120) 

c. (+(+ 10 20) (* 2 0)) 

d. (= (+ 10 20) (* 15 (+ 5 5))) 

e. + 

f* (H — h <) 

Exercise 3.3. For each question, construct a Scheme expression and evaluate it 
in DrRacket. 

a. How many seconds are there in a year? 

b. For how many seconds have you been alive? 

c. For what fraction of your life have you been in school? 
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3.5. Definitions 


Exercise 3.4. Construct a Scheme expression to calculate thè distance in inches 
that light travels during thè tinte it takes thè processor in your computer to exe- 
cute one cycle. (A meter is dehned as thè distance light travels in 1 / 299792458 f? ' 
of a second in a vacuum. Hence, light travels at 299,792,458 meters per sec- 
ond. Your processor speed is probably given in gigahertz (GHz), which are 
1,000,000,000 hertz. One hertz means once per second, so 1 GHz means thè 
processor executes 1,000,000,000 cycles per second. On a Windows machine, 
you can find thè speed of your processor by opening thè Control Panel (select it 
from thè Start menu) and selecting System. Note that Scheme performs calcu- 
lations exactly, so thè result will be displayed as a fraction. To see a more useful 
answer, use ( exact-> inexact Expression ) to convert thè value of thè expression 
to a decimai representation.) 


3.5 Definitions 

Scheme provides a simple, yet powerful, mechanism for abstraction. A defini- 
tion introduces a new name and gives it a value: 

Definition (define Name Expression) 

After a definition, thè Alarne in thè definition is now associated with thè value of 
thè expression in thè definition. A definition is not an expression since it does 
not evaluate to a value. 

A name can be any sequence of letters, digits, and special characters (such as 
-, >, ?, and !) that starts with a letter or special character. Examples of valid 
names include a, Ada, Augusta- Ada, gold49, lyuck, and yikes!\%>@\# . We don’t 
recommend using some of these names in your programs, however! A good pro- 
grammer will pick names that are easy to read, pronounce, and remember, and 
that are not easily confused with other names. 

After a name has been bound to a value by a definition, that name may be used 
in an expression: 

Expression ::=>- NameExpression 
NameExpression ::=> Name 

The value of a NameExpression is thè value associated with thè Name. (Alert 
readers should be worried that we need a more precise definition of thè meaning 
of definitions to know what it means for a value to be associated with a name. 
This informai notion will serve us well for now, but we will need a more precise 
explanation of thè meaning of a definition in Chapter 9.) 

Below we define speed- of- light to be thè speed of light in meters per second, 
define seconds-per-hour to be thè number of seconds in an hour, and use them 
to calculate thè speed of light in kilometers per hour: 

> (define speed-of-light 299792458) 

> speed-of-light 
299792458 

> (define seconds-per-hour (* 60 60)) 

> (/ (* speed-of-light seconds-per-hour) 1000) 

1079252848 4/5 
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3.6 Procedures 

In Chapter 1 we defined a procedure as a description of a process. Scheme pro- 
vides a way to define procedures that take inputs, carry out a sequence of ac- 
tions, and produce an output. Section 3.4.1 introduced some of Scheme’s prim- 
itive procedures. To construct complex programs, however, we need to be able 
to create our own procedures. 

Procedures are similar to mathematical functions in that they provide a map- 
ping between inputs and outputs, but they differ from mathematical functions 
in two important ways: 

State. In addition to producing an output, a procedure may access and mod- 
ify state. This means that even when thè same procedure is applied to thè 
same inputs, thè output produced may vary. Because mathematical func- 
tions do not have external state, when thè same function is applied to thè 
same inputs it always produces thè same result. State makes procedures 
much harder to reason about. We will ignore this issue until Chapter 9, and 
focus until then only on procedures that do not involve any state. 

Resources. Unlike an ideal mathematical function, which provides an instan- 
taneous and free mapping between inputs and outputs, a procedure re- 
quires resources to execute before thè output is produced. The most impor- 
tant resources are space (memory) and lime. A procedure may need space 
to keep track of intermediate results while it is executing. Each step of a 
procedure requires some time to execute. Predicting how long a procedure 
will take to execute and finding thè fastest procedure possible for solving 
some problem are core problems in computer Science. We consider this 
throughout this hook, and in particular in Chapter 7. 

For thè rest of this chapter, we view procedures as idealized mathematical func- 
tions: we consider only procedures that involve no state and do not worry about 
thè resources required to execute our procedures. 

3.6.1 Making Procedures 

Scheme provides a generai mechanism for making a procedure: 

Expression ::=>- ProcedureExpression 

ProcedureExpression ::=>• (lambda ( Parameters ) Expression) 

Parameters ::=>- e \ Nome Parameters 

Evaluating a ProcedureExpression produces a procedure that takes as inputs thè 
Parameters following thè lambda. The lambda special form means “make a pro- 
cedure”. The body of thè resulting procedure is thè Expression, which is not 
evaluated until thè procedure is applied. 

A ProcedureExpression can replace an Expression. This means anywhere an Ex- 
pression is used we can create a new procedure. This is very powerful since it 
means we can use procedures as inputs to other procedures and create proce- 
dures that return new procedures as their output! 

Here are some example procedures: 

(lambda (jc) (* x x)) 

Procedure that takes one input, and produces thè square of thè input value 
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higher-order 

procedure 


as its output. 

(lambda (a b ) (+ a b)) 

Procedure that takes two inputs, and produces thè sum of thè input values 
as its output. 

(lambda () 0) 

Procedure that takes no inputs, and produces 0 as its output. The result of 
applying this procedure to any argument is always 0. 

(lambda ( a ) (lambda [b] (+ a b) ) ) 

Procedure that takes one input (a), and produces as its output a procedure 
that takes one input and produces thè sum of a and that input as its output. 
This is an example of a higher-order procedure. Higher-order procedures 
produce procedures as their output or take procedures as their arguments. 
This can be confusing, but is also very powerful. 

3.6.2 Substitution Model of Evaluation 

For a procedure to be useful, we need to apply it. In Section 3.4.2, we saw thè 
syntax and evaluation rule for an ApplicationExpression when thè procedure 
to be applied is a primitive procedure. The syntax for applying a constructed 
procedure is identical to thè syntax for applying a primitive procedure: 

Expression ::=^ ApplicationExpression 

ApplicationExpression ::=>• ( Expression MoreExpressions ) 

MoreExpressions ::=>- e \ Expression MoreExpressions 

To understand how constructed procedures are evaluated, we need a new eval- 
uation rule. In this case, thè first Expression evaluates to a procedure that was 
created using a ProcedureExpression, so thè ApplicationExpression becomes: 

ApplicationExpression ::=r 

((lambda [Parameters) Expression) MoreExpressions ) 


(The underlined part is thè replacement for thè ProcedureExpression.) 

To evaluate thè application, first evaluate thè MoreExpressions in thè applica- 
tion expression. These expressions are known as thè opemnds of thè applica- 
tion. The resulting values are thè inputs to thè procedure. There must be ex- 
actly one expression in thè MoreExpressions corresponding to each name in thè 
parameters list. Next, associate thè names in thè Parameters list with thè corre- 
sponding operand values. Finally, evaluate thè expression that is thè body of thè 
procedure. Whenever any parameter name is used inside thè body expression, 
thè name evaluates to thè value of thè corresponding input that is associated 
with that name. 


Example 3.1: Square 


Consider evaluating thè following expression: 

((lambda (x) (* x x)) 2) 

It is an ApplicationExpression where thè first subexpression is thè ProcedureEx- 
pression, (lambda (x) (* x x)). To evaluate thè application, we evaluate all thè 
subexpressions and apply thè value of thè first subexpression to thè values of 
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thè remaining subexpressions. The first subexpression evaluates to a procedure 
that takes one parameter named x and has thè expression body (* x x). There is 
one operand expression, thè primitive 2, that evaluates to 2. 

To evaluate thè application we bind thè first parameter, x, to thè value of thè 
first operand, 2, and evaluate thè procedure body, (* x x). After substituting 
thè parameter values, we have (* 2 2). This is an application of thè primitive 
multiplication procedure. Evaluating thè application results in thè value 4. 

The procedure in our example, (lambda (x) (* x x)), is a procedure that takes a 
number as input and as output produces thè square of that number. We can use 
thè definition mechanism (from Section 3.5) to give this procedure a name so 
we can reuse it: 

(define square (lambda (x) (* x x))) 

This defines thè name square as thè procedure. After this, we can apply square 
to any number: 

> ( square 2) 

4 

> ( square 1 /4) 

1/16 

> ( square ( square 2)) 

16 


Example 3.2: Make adder 


The expression 

((lambda ( a ) 

(lambda (fi) (+ a fi))) 

3) 

evaluates to a procedure that adds 3 to its input. Applying that procedure to 4, 

(((lambda (a) (lambda (fi) (+ a fi))) 3) 

4) 

evaluates to 7. By using define, we can give these procedures sensible names: 

(define make-adder 
(lambda ( a ) 

(lambda (fi) (+ a fi)))) 

Then, (define add-three ( make-adder 3)) defines add-three as a procedure that 
takes one parameter and outputs thè value of that parameter plus 3. 


Abbreviated Procedure Definitions. Since we commonly define new proce- 
dures, Scheme provides a condensed notation for defining a procedure 6 : 

6 The condensed notation also includes a begin expression, which is a special forni. We will not 
need thè begin expression until we start dealing with procedures that have side effects. We describe 
thè begin special form in Chapter 9. 
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3.7. Decisions 


Definition (define ( Name Parameters ) Expression ) 

This incorporates thè lambda invisibly into thè definition, but means exactly 
thè same thing. For example, 

(define square (lambda (x) (* x x))) 

can be written equivalently as: 

(define ( square x) (* x x)) 


F xercise 3.5. Define a procedure, cube, that takes one number as input and 
produces as output thè cube of that number. 

Exercise 3.6. Define a procedure, compute-cost, that takes as input two num- 
bers, thè first represents that price of an item, and thè second represents thè 
sales tax rate. The output should be thè total cost, which is computed as thè 
price of thè item plus thè sales tax on thè item, which is its price times thè sales 
tax rate. For example, ( compute-cost 1 3 0.05) should evaluate to 1 3.65. 


3.7 Decisions 

To make more useful procedures, we need thè actions taken to depend on thè 
input values. For example, we may want a procedure that takes two numbers as 
inputs and evaluates to thè greater of thè two inputs. To define such a procedure 
we need a way of making a decision. The IfExpression expression provides a way 
of using thè result of one expression to select which of two possible expressions 
to evaluate: 

Expression ::=> IfExpression 
IfExpression ::=> (if Expression Piet j icate 
Expressionc onsequent 
Expression M tema te) 


The IfExpression replacement has three Expression terms. For clarity, we give 
each of them names as denoted by thè Predicate, Consequent, and Alternate 
subscripts. To evaluate an IfExpression, first evaluate thè predicate expression, 
Expressjonp re dicate- If it evaluates to any non-false value, thè value of thè IfEx- 
pression is thè value of Expression Consequent , thè consequent expression, and thè 
alternate expression is not evaluated at all. If thè predicate expression evaluates 
to false, thè value of thè IfExpression is thè value of Expressi on Altemate , thè alter- 
nate expression, and thè consequent expression is not evaluated at all. 

The predicate expression determines which of thè two following expressions is 
evaluated to produce thè value of thè IfExpression. If thè value of thè predicate 
is anything other than false, thè consequent expression is used. For example, if 
thè predicate evaluates to true, to a number, or to a procedure thè consequent 
expression is evaluated. 

special forni The if expression is a special forni. This means that although it looks syntacti- 
cally identical to an application (that is, it could be an application of a procedure 
named if), it is not evaluated as a normal application would be. Instead, we have 
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a special evaluation rule for if expressions. The reason a special evaluation rule 
is needed is because we do not want all thè subexpressions to be evaluated. With 
thè normal application rule, all thè subexpressions are evaluated first, and then 
thè procedure resulting from thè first subexpression is applied to thè values re- 
sulting from thè others. With thè if special form evaluation rule, thè predicate 
expression is always evaluated first and only one of thè following subexpressions 
is evaluated depending on thè result of evaluating thè predicate expression. 

This means an if expression can evaluate to a value even if evaluating one of its 
subexpressions would produce an errar. For example, 

(if (> 3 4) (* + +) 7) 

evaluates to 7 even though evaluating thè subexpression (* + +) would produce 
an errar. Because of thè special evaluation rule for if expressions, thè conse- 
quent expression is never evaluated. 


Example 3.3: Bigger 


Now that we have procedures, decisions, and definitions, we can understand thè 
bigger procedure from thè beginning of thè chapter. The definition, 

(define ( bigger a b) (if ( > a b) a b )) 
is a condensed procedure definition. It is equivalent to: 

(define bigger (lambda ( a b) (if (> ab) a b))) 

This defines thè name bigger as thè value of evaluating thè procedure expression 
(lambda ( a b ) (if (> ab) a b )). This is a procedure that takes two inputs, named 
a and b. Its body is an if expression with predicate expression (> ab). The 
predicate expression compares thè value that is bound to thè first parameter, a, 
with thè value that is bound to thè second parameter, b, and evaluates to true if 
thè value of thè first parameter is greater, and false otherwise. According to thè 
evaluation rule for an if expression, when thè predicate evaluates to any non- 
false value (in this case, true), thè value of thè if expression is thè value of thè 
consequent expression, a. When thè predicate evaluates to false, thè value of 
thè if expression is thè value of thè alternate expression, b. Hence, our bigger 
procedure takes two numbers as inputs and produces as output thè greater of 
thè two inputs. 


Exercise 3.7. Follow thè evaluation rules to evaluate thè Scheme expression: 

( bigger 3 4) 

where bigger is thè procedure defined above. (It is very tedious to follow all of 
thè steps (that’s why we normally rely on computers to do it!), but worth doing 
once to make sure you understand thè evaluation rules.) 
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Exercise 3.8. Define a procedure, xor, that implements thè logicai exclusive-or 
operation. The xor function takes two inputs, and outputs true if exactly one of 
those outputs has a true value. Otherwise, it outputs false. For example, (xor true 
true) should evaluate to false and (xor (< 3 5) (= 8 8)) should evaluate to true. 

Exercise 3.9. Define a procedure, absvalue, that takes a number as input and 
produces thè absolute value of that number as its output. For example, {ab- 
svalue 3) should evaluate to 3 and {absvalue -1 50) should evaluate to 1 50. 

Exercise 3.10. Define a procedure, bigger-magnitude, that takes two inputs, and 
outputs thè value of thè input with thè greater magnitude (that is, absolute dis- 
tance from zero). For example, {bigger-magnitude 5 -7) should evaluate to -7, 
and ( bigger-magnitude 9 -3) should evaluate to 9. 

Exercise 3.1 1. Define a procedure, biggest, that takes three inputs, and produces 
as output thè maximum value of thè three inputs. For example, ( biggest 5 7 3) 
should evaluate to 7. Find at least two different ways to define biggest, one using 
bigger, and one without using it. 


3.8 Evaluation Rules 

Here we summarize thè grammar rules and evaluation rules. Since each gram- 
olar rule has an associated evaluation rule, we can determine thè meaning of 
any grammatical Scheme fragment by combining thè evaluation rules corre- 
sponding to thè grammar rules followed to derive that fragment. 

Program e \ ProgramElement Program 

ProgramElement Expression | Defìnition 

A program is a sequence of expressions and definitions. 

Defìnition (define Nume Expression) 

A defìnition evaluates thè expression, and associates thè value of thè 
expression with thè name. 

Defìnition ::=> (define ( Name Parameters) Expression ) 

Abbreviation for 

(define Name (lambda Parameters ) Expression ) 

Expression ::=> PrimitiveExpression \ NameExpression 

| ApplicationExpression 
j ProcedureExpression | ffExpression 

The value of thè expression is thè value of thè replacement 
expression. 

PrimitiveExpression ::=f> Number \ true \ false \ primitive procedure 

Evaluation Rule 1: Primitives. A primitive expression evaluates to 
its pre-defined value. 
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NameExpression Name 

Evaluation Rule 2: Names. A name evaluates to thè value associateci 
with that name. 

ApplicationExpression ::=> ( Expression MoreExpressions ) 

Evaluation Rule 3: Application. To evaluate an application 
expression: 

a. Evaluate all thè subexpressions; 

b. Then, apply thè value of thè first subexpression to thè values of 
thè remaining subexpressions. 

MoreExpressions ::=^ e \ Expression MoreExpressions 

ProcedureExpression (lambda ( Parameters ) Expression') 

Parameters ::=^> e \ Name Parameters 

Evaluation Rule 4: Lambda. Lambda expressions evaluate to a 
procedure that takes thè given parameters and has thè expression as 
its body. 

IfExpression (if Expression Prp r | i ratP 

Expression Consequent 
Expression | t ema t e ) 


Evaluation Rule 5: If. To evaluate an if expression, (a) evaluate thè 
predicate expression; then, (b) if thè value of thè predicate 
expression is a false value then thè value of thè if expression is thè 
value of thè alternate expression; otherwise, thè value of thè if 
expression is thè value of thè consequent expression. 

The evaluation rule for an application (Rule 3b) uses apply to perform thè ap- 
plication. Apply is defined by thè two application rules: 

Application Rule 1: Primitives. 

To apply a primitive procedure, just do it. 

Application Rule 2: Constructed Procedures. 

To apply a constructed procedure, evaluate thè body of thè procedure with 
each parameter name bound to thè corresponding input expression value. 

Application Rule 2 uses thè evaluation rules to evaluate thè expression. Thus, 
thè evaluation rules are defined using thè application rules, which are defined 
using thè evaluation rules! This appears to be a circular defìnition, but as with 
thè grammar examples, it has a base case. Some expressions evaluate without 
using thè application rules (e.g., primitive expressions, name expressions), and 
some applications can be performed without using thè evaluation rules (when 
thè procedure to apply is a primitive). Hence, thè process of evaluating an ex- 
pression will sometimes finish and when it does we end with thè value of thè 
expression. 7 


7 This does not guarantee that evaluation cilwciys finishes, however! The next chapter includes 
some examples where evaluation never finishes. 
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3.9. Summary 


3.9 Summary 

At this point, we have covered enough of Scheme to write useful programs (even 
if thè programs we have seen so far seem rather dull) . In fact (as we show in 
Chapter 12), we have covered enough to express every possible computation! 
We just need to combine these constructs in more complex ways to perform 
more interesting computations. The next chapter (and much of thè rest of this 
hook), focuses on ways to combine thè constructs for making procedures, mak- 
ing decisions, and applying procedures in more powerful ways. 


4 

Problems and Procedures 


A great discovery solves a great problem, but there is a grain ofdiscovery in thè solution of 
any problem. Your problem may be modest, but ifit challenges your curiosity and brings into 
playyour inventive faculties, and ifyou solve it byyour own means, you may experience thè 

tension and enjoy thè triumph ofdiscovery. 

George Pólya, How to Solve It 

Computers are tools for performing computations to solve problems. In this 
chapter, we consider what it means to solve a problem and explore some strate- 
gies for constructing procedures that solve problems. 

4.1 Solving Problems 

Traditionally, a problem is an obstacle to overcome or some question to answer. 

Once thè question is answered or thè obstacle circumvented, thè problem is 
solved and we can declare victory and move on to thè next one. 

When we talk about writing programs to solve problems, though, we have a 
larger goal. We don’t just want to solve one instance of a problem, we want an 
algorithm that can solve all instances of a problem. A problem is defined by problem 
its inputs and thè desired property of thè output. Recali from Chapter 1, that a 
procedure is a precise description of a process and a procedure is guaranteed 
to always finish is called an algorithm. The name algorithm is a Latinization 
of thè name of thè Persian mathematician and scientist, Muhammad ibn Musa 
al-Khwàrizml, who published a hook in 825 on calculation with Hindu numer- 
als. Although thè name algorithm was adopted after al-Khwàrizrm’s hook, algo- 
rithms go back much further than that. The ancient Babylonians had algorithms 
for fìnding square roots more than 3500 years ago (see Exploration 4.1). 

For example, we don’t just want to fìnd thè best route between New York and 
Washington, we want an algorithm that takes as inputs thè map, start location, 
and end location, and outputs thè best route. There are infìnitely many possible 
inputs that each specify different instances of thè problem; a generai solution to 
thè problem is a procedure that finds thè best route for all possible inputs. 1 

To defìne a procedure that can solve a problem, we need to define a procedure 
that takes inputs describing thè problem instance and produces a different in- 
formation process depending on thè actual values of its inputs. A procedure 

^ctually fìnding a generai algorithm that does without needing to essentially try all possible 
routes is a challenging and interesting problem, for which no effìcient solution is known. Find- 
ing one (or proving no fast algorithm exists) would resolve thè most important open problem in 
computer Science! 
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4.2. Composing Procedures 


takes zero or more inputs, and produces one output or no outputs 2 , as shown in 
Figure 4.1. 



Output 


Figure 4. 1. A procedure maps inputs to an output. 


Our goal in solving a problem is to devise a procedure that takes inputs that 
define a problem instance, and produces as output thè solution to that problem 
instance. The procedure should be an algorithm — this means every application 
of thè procedure must eventually finish evaluating and produce an output value. 

There is no magic wand for solving problems. But, most problem solving in- 
volves breaking problems you do not yet know how to solve into simpler and 
simpler problems until you find problems simple enough that you already know 
how to solve them. The creative challenge is to find thè simpler subproblems 
that can be combined to solve thè originai problem. This approach of solving 
problems by breaking them into simpler parts is known as divide-and-conquer. 

divide-and-conquer 

The following sections describe a two key forms of divide-and-conquer problem 
solving: composition and recursive problem solving. We will use these same 
problem-solving techniques in different forms throughout this book. 

4.2 Composing Procedures 

One way to divide a problem is to split it into steps where thè output of thè first 
step is thè input to thè second step, and thè output of thè second step is thè 
solution to thè problem. Each step can be defined by one procedure, and thè two 
procedures can be combined to create one procedure that solves thè problem. 

Figure 4.2 shows a composition of two functions, / and g. The output of / is 
used as thè input to g. 



Figure 4.2. Composition. 


We can express this composition with thè Scheme expression (g (/ x)) where x 
is thè input. The written order appears to be reversed from thè picture in Fig- 
ure 4.2. This is because we apply a procedure to thè values of its subexpressions: 

2 Although procedures can produce more than one output, we limit our discussion here to proce- 
dures that produce no more than one output. In thè next chapter, we introduce ways to construct 
complex data, so any number of output values can be packaged into a single output. 
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thè values of thè inner subexpressions must be computed first, and then used as 
thè inputs to thè outer applications. So, thè inner subexpression (/ x} is evalu- 
ated first since thè evaluation rule for thè outer application expression is to first 
evaluate all thè subexpressions. 

To define a procedure that implements thè composed procedure we make x a 
parameter: 

(define fog (lambda (x) (g (f x)))) 

This defines fog as a procedure that takes one input and produces as output thè 
composition of / and g applied to thè input parameter. This works for any two 
procedures that both take a single input parameter. 

We can compose thè square and cube procedures from Chapter 3: 

(define sixth-power (lambda (x) ( cube ( square x)))) 

Then, ( sixth-power 2) evaluates to 64. 

4.2.1 Procedures as Inputs and Outputs 

All thè procedure inputs and outputs we have seen so far have been numbers. 
The subexpressions of an application can be any expression including a proce- 
dure. A higher-order procedure is a procedure that takes other procedures as in- 
puts or that produces a procedure as its output. Higher-order procedures give us 
thè ability to write procedures that behave differently based on thè procedures 
that are passed in as inputs. 

We can create a generic composition procedure by making/ and g parameters: 
(define fog (lambda (/ g x) (g (/ x)))) 

The fog procedure takes three parameters. The first two are both procedures 
that take one input. The third parameter is a value that can be thè input to thè 
first procedure. 

For example, ( fog square cube 2) evaluates to 64, and ( fog (lambda (x) (+ x 1)) 
square 2) evaluates to 9. In thè second example, thè first parameter is thè proce- 
dure produced by thè lambda expression (lambda (x) (+ x 1)). This procedure 
takes a number as input and produces as output that number plus one. We use 
a defini tion to name this procedure ine (short for incrementi: 

(define ine (lambda (x) (+ x 1))) 

A more useful composition procedure would separate thè input value, x, from 
thè composition. The feompose procedure takes two procedures as inputs and 
produces as output a procedure that is their composition: 3 

(define feompose 

(lambda (/ gl (lambda (x) (g (/ x))))) 

The body of thè feompose procedure is a lambda expression that makes a proce- 
dure. Hence, thè result of applying feompose to two procedures is not a simple 
value, but a procedure. The resulting procedure can then be applied to a value. 


3 We name our composition procedure feompose to avoid collision with thè built-in compose pro- 
cedure that behaves similarly. 


higher-order 

procedure 
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Here are some examples using /compose: 

> (ft compose ine ine) 

#<procedure> 

> ((f compose ine ine) 1) 

3 

> ((feompose ine square ) 2) 

9 

> ((fi compose square ine) 2) 

5 


Exercise 4.1. For each expression, give thè value to which thè expression evalu- 
ates. Assume feompose and ine are defined as above. 

a. ((f compose square square) 3) 

b. (feompose (lambda (x) (* x 2)) (lambda (x) (/ x 2))) 

c. ((feompose (lambda (x) (* x 2)) (lambda (x) (/ x 2))) 1120) 

d. ((feompose (feompose ine ine) ine) 2) 

Exercise 4.2. Suppose we define self-compose as a procedure that composes a 
procedure with itself: 

(define (self-compose f) (feompose f f)) 

Explain how (((feompose self-compose self-compose) ine) 1) is evaluated. 

Exercise 4.3. Define a procedure fcompose3 that takes three procedures as in- 
put, and produces as output a procedure that is thè composition of thè three 
input procedures. For example, (( fcompose3 abs ine square) -5) should evalu- 
ate to 36. Define fcompose3 two different ways: once without using feompose, 
and once using feompose. 

Exercise 4.4. The feompose procedure only works when both input procedures 
take one input. Define a f2compose procedure that composes two procedures 
where thè first procedure takes two inputs, and thè second procedure takes one 
input. For example, ((f2compose + abs) 3 -5) should evaluate to 2. 


4.3 Recursive Problem Solving 

In thè previous section, we used functional composition to break a problem into 
two procedures that can be composed to produce thè desired output. A partic- 
ularly useful variation on this is when we can break a problem into a smaller 
version of thè originai problem. 

The goal is to be able to feed thè output of one application of thè procedure 
back into thè same procedure as its input for thè next application, as shown in 
Figure 4.3. 

Here’s a corresponding Scheme procedure: 

(define / (lambda (n) (f ri))) 
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/ 



Input 


Figure 4.3. Circular Composition. 


Of course, this doesn’t work very well! 4 Every application off results in another 
application off to evaluate. This never stops — no output is ever produced and 
thè interpreter will keep evaluating applications of / until it is stopped or runs 
out of memory. 

We need a way to make progress and eventually stop, instead of going around 
in circles. To make progress, each subsequent application should have a smaller 
input. Then, thè applications stop when thè input to thè procedure is simple 
enough that thè output is already known. The stopping condition is called thè 
base case, similarly to thè grammar rules in Section 2.4. In our grammar ex- base case 
amples, thè base case involved replacing thè nonterminal with nothing (e.g., 
MoreDigits e) or with a terminal (e.g., Noun ::=> Alice). In recursive pro- 
cedures, thè base case will provide a solution for some input for which thè prob- 
lem is so simple we already know thè answer. When thè input is a number, this 
is often (but not necessarily) when thè input is 0 or 1 . 

To define a recursive procedure, we use an if expression to test if thè input matches 
thè base case input. If it does, thè consequent expression is thè known answer 
for thè base case. Otherwise, thè recursive case applies thè procedure again but 
with a smaller input. That application needs to make progress towards reaching 
thè base case. This means, thè input has to change in a way that gets closer to 
thè base case input. If thè base case is for 0, and thè originai input is a positive 
number, one way to get closer to thè base case input is to subtract 1 from thè 
input value with each recursive application. 

This evaluation spirai is depicted in Figure 4.4. With each subsequent recursive 
cali, thè input gets smaller, eventually reaching thè base case. For thè base case 
application, a result is returned to thè previous application. This is passed back 
up thè spirai to produce thè final output. Keeping track of where we are in a 
recursive evaluation is similar to keeping track of thè subnetworks in an RTN 
traversai. The evaluator needs to keep track of where to return after each recur- 
sive evaluation completes, similarly to how we needed to keep track of thè stack 
of subnetworks to know how to proceed in an RTN traversai. 

Here is thè corresponding procedure: 

(define g 
(lambda (n) 

(if (= n 0) 1 (g(- ni))))) 

Unlike thè earlier circular / procedure, if we apply g to any non-negative integer 
it will eventually produce an output. For example, consider evaluating (g 2). 

4 Curious readers should try entering this definitimi into a Scheme interpreter and evaluating (f 
0). If you get tired of waiting for an output, in DrRacket you can click thè Stop button in thè upper 
right corner to interrupt thè evaluation. 
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Figure 4.4. Recursive Composition. 


When we evaluate thè fìrst application, thè value of thè parameter n is 2, so thè 
predicate expression (= n 0) evaluates to false and thè value of thè procedure 
body is thè value of thè alternate expression, (g(— ni)). The subexpression, (— n 
1 ) evaluates to 1 , so thè result is thè result of applying g to 1 . As with thè previous 
application, this leads to thè application, (g [- n 1)), but this time thè value of 
n is 1 , so (— ni) evaluates to 0. The next application leads to thè application, (g 
0). This time, thè predicate expression evaluates to true and we have reached thè 
base case. The consequent expression is just 1 , so no further applications of g 
are performed and this is thè result of thè application (g 0). This is returned as 
thè result of thè (g 1 ) application in thè previous recursive cali, and then as thè 
output of thè originai (g 2) application. 



We can think of thè recursive evaluation as winding until thè base case is reached, 
and then unwinding thè outputs back to thè originai application. For this pro- 
cedure, thè output is not very interesting: no matter what positive number we 
apply g to, thè eventual result is 1 . To solve interesting problems with recursive 
procedures, we need to accumulate results as thè recursive applications wind 
or unwind. Examples 4.1 and 4.2 illustrate recursive procedures that accumu- 
late thè result during thè unwinding process. Example 4.3 illustrates a recursive 
procedure that accumulates thè result during thè winding process. 


Example 4.1: Factorial 


How many different arrangements are there of a deck of 52 playing cards? 

The top card in thè deck can be any of thè 52 cards, so there are 52 possible 
choices for thè top card. The second card can be any of thè cards except for 
thè card that is thè top card, so there are 51 possible choices for thè second card. 
The third card can be any of thè 50 remaining cards, and so on, until thè last card 
for which there is only one choice remaining. 


52 * 51 * 50 * ■■■ * 2 * 1 


factorial This is known as thè factorial function (denoted in mathematics using thè ex- 
clamation point, e.g., 52!). It can be defìned recursively: 


0 ! = 1 

ni — n* (n — 1)! for all n > 0 

The mathematical defmition of factorial is recursive, so it is naturai that we can 
defìne a recursive procedure that computes factorials: 

(define ( factorial n) 

(if (= n 0) 

1 

(* n ( factorial (— n 1))))) 
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Evaluating ( factorial 52) produces thè number of arrangements of a 52-card deck: 
a sixty-eight digit number starting with an 8. 

Th e factorial procedure has structure very similar to our earlier definition of thè 
useless recursive g procedure. The only difference is thè alternative expression 
for thè if expression: in g we used (g (— n 1)); in factorial we added thè outer 
application of *: (* n ( factorial (— n 1))). Instead of just evaluating to thè result 
of thè recursive application, we are now combining thè output of thè recursive 
evaluation with thè input n using a multiplication application. 


Exercise 4.5. How many different ways are there of choosing an unordered 5- 
card hand from a 52-card deck? 

This is an instance of thè “n choose k" problem (also known as thè binomial 
coefficienti : how many different ways are there to choose a set of k items from 
n items. There are n ways to choose thè first item, n — 1 ways to choose thè 
second, . . ., and n — k + 1 ways to choose thè k th item. But, since thè order does 
not matter, some of these ways are equivalent. The number of possible ways to 
order thè k items is k!, so we can compute thè number of ways to choose k items 
from a set of n items as: 

n * (n — 1) * • • • * (n — k + 1) n\ 

k\ ( n — k)\k\ 


a. Define a procedure choose that takes two inputs, n (thè size of thè item set) 
and k (thè number of items to choose), and outputs thè number of possible 
ways to choose k items from n. 

b. Compute thè number of possible 5-card hands that can be dealt from a 52- 
card deck. 

c. [*] Compute thè likelihood of being dealt aflush (5 cards all of thè same sufi). 
In a standard 52-card deck, there are 13 cards of each of thè four suits. Hint: 
divide thè number of possible flush hands by thè number of possible hands. 


Exercise 4.6. Reputedly, when Karl Gauss was in elementary school his teacher 
assigned thè class thè task of summing thè integers from 1 to 100 (e.g., 1+2 + 
3 + ■ • • + 100) to keep them busy. Being thè (future) “Prince of Mathematics”, 
Gauss developed thè formula for calculating this sum, that is now known as thè 
Gauss sum. Had he been a computer scientist, however, and had access to a 
Scheme interpreter in thè late 1700s, he might have instead defined a recursive 
procedure to solve thè problem. Define a recursive procedure, gauss-sum, that 
takes a number n as its input parameter, and evaluates to thè sum of thè integers 
from 1 to n as its output. For example, ( gauss-sum 1 00) should evaluate to 5050. 



Exercise 4.7. [*] Define a higher-order procedure, accumulate, that can be used 
to make both gauss-sum (from Exercise 4.6) and factorial. The accumulate pro- 
cedure should take as its input thè function used for accumulation (e.g., * for 
factorial, + for gauss-sum ). With your accumulate procedure, (( accumulate +) 
100) should evaluate to 5050 and (( accumulate *) 3) should evaluate to 6. We as- 
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sume thè result of thè base case is 1 (although a more generai procedure could 
take that as a parameter) . 

Hint: since your procedure should produce a procedure as its output, it could 
start like this: 

(define [accumulate f) 

(lambda [ri) 

(if (— n 1 ) 1 


Example 4.2: Find Maximum 


Consider thè problem of defming a procedure that takes as its input a procedure, 
a low value, and a high value, and outputs thè maximum value thè input proce- 
dure produces when applied to an integer value between thè low value and high 
value input. We name thè inputs f, low, and high. To find thè maximum, thè 
find-maximum procedure should evaluate thè input procedure / at every inte- 
ger value between thè low and high, and output thè greatest value found. 

Here are a few examples: 

> [find-maximum (lambda (x) x) 1 20) 

20 

> [find-maximum (lambda (x) (— 10 x)) 1 20) 

9 

> [find-maximum (lambda (x) (* x (— 10 x))) 1 20) 

25 

To define thè procedure, think about how to combine results from simpler prob- 
lems to find thè result. For thè base case, we need a case so simple we already 
know thè answer. Consider thè case when low and high are equal. Then, there 
is only one value to use, and we know thè value of thè maximum is [f low ) . So, 
thè base case is (if [— low high) [f low) ...). 

How do we make progress towards thè base case? Suppose thè value of high is 
equal to thè value of low plus 1. Then, thè maximum value is either thè value of 
(/ low) or thè value of (/ (+ low 1 )) . We could select it using thè higger procedure 
(from Example 3.3): [bigger [f low) [f (+ low 1))). We can extend this to thè case 
where high is equal to low plus 2: 

[bigger [f low) [bigger [f (+ low 1)) (/ (+ low 2)))) 

The second operand for thè outer bigger evaluation is thè maximum value of thè 
input procedure between thè low value plus one and thè high value input. If we 
name thè procedure we are defming find-maximum, then this second operand 
is thè result of [find-maximum f (+ low 1) high). This works whether high is 
equal to (+ low 1 ), or (+ low 2), or any other value greater than high. 

Putting things together, we have our recursive definition of find-maximum: 

(define [find-maximum f low high) 

(if (= low high) 

[f low) 

[bigger [f low) [find-maximum f (+ low 1) high)))) 

Exercise 4.8. To find thè maximum of a function that takes a reai number as 
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its input, we need to evaluate at all numbers in thè range, not just thè integers. 
There are infinitely many numbers between any two numbers, however, so this 
is impossible. We can approximate this, however, by evaluating thè function at 
many numbers in thè range. 

Define a procedure find-maximum-epsilon that takes as input a function /, a 
low range value low, a high range value high , and an increment epsilon, and 
produces as output thè maximum value off in thè range between low and high 
at interval epsilon. As thè value of epsilon decreases, find-maximum-epsilon 
should evaluate to a value that approaches thè actual maximum value. 

For example, 

( find-maximum-epsilon (lambda (jc) (* x (— 5.5 x))) 110 1) 
evaluates to 7.5. And, 

( find-maximum-epsilon (lambda (x) (* x (— 5.5 x))) 1 10 0.01) 
evaluates to 7.5625. 


Exercise 4.9. [*] The find-maximum procedure we defined evaluates to thè 
maximum value of thè input function in thè range, but does not provide thè 
input value that produces that maximum output value. Define a procedure 
that finds thè input in thè range that produces thè maximum output value. 
For example, ( find-maximum-input ine 1 10) should evaluate to 10 and [find- 
maximum-input (lambda (x) (* x (— 5.5 x))) 1 10) should evaluate to 3. 

Exercise 4.10. [*] Defìne a find-area procedure that takes as input a function 
/, a low range value low, a high range value high, and an increment epsilon, 
and produces as output an estimate for thè area under thè curve produced by 
thè function / between low and high using thè epsilon value to determine how 
many regions to evaluate. 


Example 4.3: Euclid’s Algorithm 


In Book 7 of thè Elements, Euclid describes an algorithm for finding thè greatest 
common divisor of two non-zero integers. The greatest common divisor is thè 
greatest integer that divides both of thè input numbers without leaving any re- 
mainder. For example, thè greatest common divisor of 150 and 200 is 50 since 
(/ 150 50) evaluates to 3 and (/ 200 50) evaluates to 4, and there is no number 
greater than 50 that can evenly divide both 150 and 200. 

The modulo primitive procedure takes two integers as its inputs and evaluates to 
thè remainder when thè first input is divided by thè second input. For example, 
( modulo 6 3) evaluates to 0 and ( modulo 7 3) evaluates to 1 . 

Euclid’s algorithm stems from two properties of integers: 

1. If ( modulo a b ) evaluates to 0 then b is thè greatest common divisor of a 
and b. 

2. If ( modulo a b ) evaluates to a non-zero integer r, thè greatest common 
divisor of a and b is thè greatest common divisor of b and r. 

We can define a recursive procedure for finding thè greatest common divisor 
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closely following Euclid’s algorithm 5 : 

(define ( gcd-euclid a b) 

(if (= ( modulo ab) 0) b ( gcd-euclid b ( modulo a b )))) 

The structure of thè definition is similar to thè factorial definition: thè proce- 
dure body is an if expression and thè predicate tests for thè base case. For thè 
gcd-euclid procedure, thè base case corresponds to thè first property above. It 
occurs when b divides a evenly, and thè consequent expression is b. The alter- 
nate expression, ( gcd-euclid b ( modulo a b)), is thè recursive application. 

The gcd-euclid procedure differs from thè factorial definition in that there is no 
outer application expression in thè recursive cali. We do not need to combine 
thè result of thè recursive application with some other value as was done in thè 
factorial definition, thè result of thè recursive application is thè final result. Un- 
like thè factorial and fìnd-maximum examples, thè gcd-euclid procedure pro- 
duces thè result in thè base case, and no further computation is necessary to 
produce thè final result. When no further evaluation is necessary to get from 
thè result of thè recursive application to thè final result, a recursive definition is 
tail recursive said to be tail recursive. Tail recursive procedures have thè advantage that they 
can be evaluated without needing to keep track of thè stack of previous recur- 
sive calls. Since thè final cali produces thè final result, there is no need for thè 
interpreter to unwind thè recursive calls to produce thè answer. 


Exercise 4.1 1. Show thè structure of thè gcd-euclid applications in evaluating 
(, gcd-euclid 6 9). 

Exercise 4.12. Provide a convincing argument why thè evaluation of ( gcd- 
euclid a b) will always finish when thè inputs are both positive integers. 

Exercise 4.13. Provide an alternate definition of factorial that is tail recursive. 
To be tail recursive, thè expression containing thè recursive application cannot 
be part of another application expression. (Hint: define a factorial-helper proce- 
dure that takes an extra parameter, and then define factorial as (define ( factorial 
ri) ( factorial-helper ni)).) 

Exercise 4.14. Provide a tail recursive definition of flnd-maximum. 

Exercise 4.15. [★*] Provide a convincing argument why it is possible to trans- 
form any recursive procedure into an equivalent procedure that is tail recursive. 


Exploration4.1: SquareRoots 


One of thè earliest known algorithms is a method for computing square roots. It 
is known as Heron’s method after thè Greek mathematician Heron of Alexandria 
who lived in thè first century AD who described thè method, although it was also 
known to thè Babylonians many centuries earlier. Isaac Newton developed a 


5 DrRacket provides a built-in procedure gcd that computes thè greatest common divisor. We 
name our procedure gcd-euclid to avoid a clash with thè build-in procedure. 
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more generai method for estimating functions based on their derivatives known 
as Netwon’s method, ofwhich Heron’s method is a specialization. 

Square root is a mathematical function that take a number, a, as input and out- 
puts a value x such that x 2 — a. For many numbers (including 2), thè square 
root is irrational, so thè best we can hope for with is a good approximation. We 
define a procedure find-sqrt that takes thè target number as input and outputs 
an approximation for its square root. 


Heron’s method works by starting with an arbitrary guess, go- Then, with each 
iteration, compute a new guess (g n is thè n th guess) that is a function of thè pre- 
vious guess (gn-ì ) and thè target number (a): 


As n increases g n gets closer and closer to thè square root of a. 


The defmition is recursive since we compute g n as a function of g n -\, so we can 
define a recursive procedure that computes Heron’s method. First, we defìne a 
procedure for computing thè next guess from thè previous guess and thè target: 


(define [heron-next- guess a g ) (/ (+ g (/ a g)) 2)) 



Heron of 
Alexandria 


Next, we defìne a recursive procedure to compute thè n th guess using Heron’s 
method. It takes three inputs: thè target number, a, thè number of guesses to 
make, n, and thè value of thè fìrst guess, g. 

(define ( heron-method an g) 

(if {— n 0) 
g 

( heron-method a [— n 1) ( heron-next-guess a g)))) 


To start, we need a value for thè fìrst guess. The choice doesn’t really matter 
— thè method works with any starting guess (but will reach a closer estimate 
quicker if thè starting guess is good). We will use 1 as our starting guess. So, we 
can defìne a find-sqrt procedure that takes two inputs, thè target number and 
thè number of guesses to make, and outputs an approximation of thè square 
root of thè target number. 


(define ( find-sqrt a guesses ) 

( heron-method a guesses 1)) 


Heron’s method converges to a good estimate very quickly: 

> ( square ( find-sqrt 2 0)) 

1 

> ( square [find-sqrt 2 1)) 

2 1/4 

> ( square [find-sqrt 2 2)) 

2 1/144 

> [square [find-sqrt 2 4)) 

2 1/221682772224 

> [exact-> inexact [find-sqrt 2 5)) 

1.4142135623730951 
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The actual square root of 2 is 1.414213562373095048 . . so our estimate is correct 
to 16 digits after only five guesses. 

Users of square roots don’t really care about thè method used to find thè square 
root (or how many guesses are used). Instead, what is important to a square root 
user is how dose thè estimate is to thè actual value. Can we change our flnd-sqrt 
procedure so that instead of taking thè number of guesses to make as its second 
input it takes a minimum tolerance value? 

Since we don’t know thè actual square root value (otherwise, of course, we could 
just return that), we need to measure tolerance as how dose thè square of thè 
approximation is to thè target number. Hence, we can stop when thè square of 
thè guess is dose enough to thè target value. 

(define ( close-enough ? a tolerance g) 

[<— ( abs (- a [square g))) tolerance )) 

The stopping condition for thè recursive defìnition is now when thè guess is 
dose enough. Otherwise, our defìnitions are thè same as before. 

(define [heron-method-tolerance a tolerance g ) 

(if [close-enough? a tolerance g) 
g 

[heron-method-tolerance a tolerance [heron-next-guess a g)))) 

(define [flnd-sqrt-approx a tolerance) 

[heron-method-tolerance a tolerance 1)) 

Note that thè value passed in as tolerance does not change with each recursive 
cali. We are making thè problem smaller by making each successive guess closer 
to thè required answer. 

Here are some example interactions with flnd-sqrt-approx: 

> [exact-> inexact [square [flnd-sqrt-approx 2 0.01))) 

2.0069444444444446 

> [exact-> inexact [square [flnd-sqrt-approx 2 0.0000001))) 

2.000000000004511 

a. How accurate is thè built-in sqrt procedure? 

b. Can you produce more accurate square roots than thè built-in sqrt proce- 
dure? 

c. Why doesn’t thè built-in procedure do better? 


4.4 Evaluating Recursive Applications 

Evaluating an application of a recursive procedure follows thè evaluation rules 
just like any other expression evaluation. It may be confusing, however, to see 
that this works because of thè apparent circularity of thè procedure defìnition. 

Here, we show in detail thè evaluation steps for evaluating [factorial 2) . The eval- 
uation and application rules refer to thè rules summary in Section 3.8. We first 
show thè complete evaluation following thè substitution model evaluation rules 
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in full gory detail, and later review a subset showing thè most revealing steps. 
Stepping through even a fairly simple evaluation using thè evaluation rules is 
quite tedious, and not something humans should do very often (that’s why we 
have computers!) but instructive to do once to understand exactly how an ex- 
pression is evaluated. 

The evaluation rule for an application expression does not specify thè order in 
which thè subexpressions are evaluated. A Scheme interpreter is free to evaluate 
them in any order. Here, we choose to evaluate thè subexpressions in thè order 
that is most readable. The value produced by an evaluation does not depend on 
thè order in which thè subexpressions are evaluated. 6 

In thè evaluation steps, we use typewriter font for uninterpreted Scheme ex- 
pressions and sans-serif font to show values. So, 2 represents thè Scheme expres- 
sion that evaluates to thè number 2. 


(factorial 2) Evaluation Rule 3 (a) : Application subexpressions 

(factorial 2) Evaluation Rule 2: Name 


( (lambda (n) (if (= n 0) 1 (* n (factorial (- n 1))))) 2) 

Evaluation Rule 4: Lambda 

((lambda (n) (if (= n 0) 1 (* n (factorial (- n 1))))) 2) Evaluation Rule 1: Primitive 

((lambda (n) (if (= n 0) 1 (* n (factorial (- n 1))))) 2) 

Evaluation Rule 3(b): Application, Application Rule 2 
(if (= 2 0) 1 (* 2 (factorial (- 2 1)))) Evaluation Rule 5 (a): If predicate 


(if (= 2 0) 1 (* 2 (factorial (- 2 1)))) 

Evaluation Rule 3 (a): Application subexpressions 
(if (=2 0) 1 (* 2 (factorial (- 2 1)))) Evaluation Rule 1: Primitive 


(if (= 2 0) 1 (* 2 (factorial (■ 

(if false 1 (* 2 (factorial (- 2 
(* 2 (factorial (- 2 1))) 

(* 2 (factorial (- 2 1))) 

(* 2 (factorial (- 2 1))) 


(* 2 (factorial (- 2 1))) 
(* 2 (factorial (- 2 y) ) ) 
(* 2 (factorial (- 2 1))) 
(* 2 ( factorial 1)) 


■ 2 1 )))) 

Evaluation Rule 3(b): Application, Application Rule 1 
1) ) ) ) Evaluation Rule 5(b): If alternate 

EraiuationTTuTc arai : A pp I i catìon s uncx n tyy i o n s 
Evaluation Rule 1: Primitive 
Evaluation Rule 3 (a): Application subexpressions 
Evaluation Rule 3 (a): Application subexpressions 
Evaluation Rule 1: Primitive 
Evaluation Rule 3(b): Application, Application Rule 1 
Continue Evaluation Rule 3(a); Evaluation Rule 2: Name 


(* 2 ((lambda (n) (if (= n 0) 1 (* n (factorial (- n 1))))) 1)) 


Evaluation Rule 4: Lambda 


(* 2 ((lambda (n) (if (= n 0) 1 (* n (factorial (- n 1))))) 1)) 

Evaluation Rule 3(b): Application, Application Rule 2 

(* 2 (if (= 1 0) 1 (* 1 (factorial (- 1 1)))) ) 

Evaluation Rule 5 (a): If predicate 

(* 2 (if (= 1 0) 1 (* 1 (factorial (- 1 1))))) 

Evaluation Rule 3 (a): Application subexpressions 
(* 2 (if (= 1 0) 1 (* 1 (factorial (- 1 1))))) 

Evaluation Rule 1: Primitives 


(* 2 (if (= 1 0) 1 (* 1 (factorial (- 1 1))))) 

Evaluation Rule 3(b): Application Rule 1 

(* 2 (if false 1 (* 1 (factorial (- 1 1)))) ) 

Evaluation Rule 5(b): If alternate 
(* 2 (* 1 (factorial (-1 1))) ) Evaluation Rule 3 (a) : Application 


6 This is only true for thè subset of Scheme we have defìned so far. Once we introduce side effects 
and mutation, it is no longer thè case, and expressions can produce different results depending on 
thè order in which they are evaluated. 
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2 


(* 2 (* 1 (factorial (-1 1)))) 
(* 2 (* 1 (factorial (-1 1)) )) 

(* 2 (* 1 (factorial (- 1 1) ) ) ) 

(* 2 (* 1 (factorial (- 1 y) ) ) ) 

(* 2 (* 1 (factorial (- 1 1)))) 


Evaluation Rule 1: Primitives 
Evaluation Rule 3 (a): Application 
Evaluation Rule 3 (a): Application 
Evaluation Rule 1: Primitives 


Evaluation Rule 3(b): Application, Application Rule 1 
(* 2 (* 1 ( factorial 0))) Evaluation Rule 2: Name 

(* 2 (* 1 ( (lambda (n) (if (= n 0) 1 (* n (fact... )))) 0))) 

Evaluation Rule 4, Lambda 


(* 2 (* 1 ((lambda (n) (if (= n 0) 1 (* n (factorial (- n 1))))) 0) )) 

Evaluation Rule 3 (b), Application Rule 2 

(* 2 (* 1 (if (= 0 0) 1 (* 0 (factorial (- 0 1)))) )) 

Evaluation Rule 5 (a): If predicate 

(* 2 (* 1 (if (= 0 0) 1 (* 0 (factorial (- 0 1)))))) 

Evaluation Rule 3 (a): Application subexpressions 
(* 2 (* 1 (if (= 0 0) 1 (* 0 (factorial (- 0 1)))))) 

Evaluation Rule 1: Primitives 


(* 2 (* 1 (if (= 0 0) 1 (* 0 (factorial (- 0 1)))))) 

Evaluation Rule 3(b): Application, Application Rule 1 
(* 2 (* 1 (if true 1 (* 0 (factorial (- 0 1)))) )) 

Evaluation Rule 5(b): If consequent 
(* 2 (* 1 ì)) Evaluation Rule 1: Primitives 

(* 2 (* 1 1) ) Evaluation Rule 3 (b): Application, Application Rule 1 

(* 2 1 ) Evaluation Rule 3 (b) : Application, Application Rule 1 

Evaluation flnished, no unevaluated expressions remain. 


The key to evaluating recursive procedure applications is if special evaluation 
rule. If thè if expression were evaluated like a regular application all subexpres- 
sions would be evaluated, and thè alternative expression containing thè recur- 
sive cali would never finish evaluating! Since thè evaluation rule for if evaluates 
thè predicate expression first and does not evaluate thè alternative expression 
when thè predicate expression is true, thè circularity in thè definition ends when 
thè predicate expression evaluates to true. This is thè base case. In thè example, 
this is thè base case where (— n 0) evaluates to true and instead of producing 
another recursive cali it evaluates to 1 . 


The Evaluation Stack. The structure of thè evaluation is clearer from just thè 
most revealing steps: 

(factorial 2) 

(* 2 (factorial 1)) 

(* 2 (* 1 (factorial 0))) 

(* 2 (* 1 1 )) 

(* 2 1 ) 

2 


Step 1 starts evaluating ( factorial 2). The result is found in Step 42. To eval- 
uate ( factorial 2), we follow thè evaluation rules, eventually reaching thè body 
expression of thè if expression in thè factorial definition in Step 17. Evaluating 
this expression requires evaluating thè (factorial 1) subexpression. At Step 17, 
thè first evaluation is in progress, but to complete it we need thè value resulting 
from thè second recursive application. 

Evaluating thè second application results in thè body expression, (* 1 (factorial 
0)), shown for Step 31. At this point, thè evaluation of (factorial 2) is stuck in 
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Evaluation Rule 3, waiting for thè value of ( factorial 1) subexpression. The eval- 
uation of thè ( factorial 1) application leads to thè ( factorial 0) subexpression, 
which must be evaluated before thè ( factorial 1) evaluation can complete. 

In Step 40, thè ( factorial 0) subexpression evaluation has completed and pro- 
duced thè value 1 . Now, thè ( factorial 1 ) evaluation can complete, producing 1 
as shown in Step 41. Once thè ( factorial 1) evaluation completes, all thè subex- 
pressions needed to evaluate thè expression in Step 17 are now evaluated, and 
thè evaluation completes in Step 42. 

Each recursive application can be tracked using a stack, similarly to process- 
ing RTN subnetworks (Section 2.3). A stack has thè property that thè first item 
pushed on thè stack will be thè last item removed — all thè items pushed on 
top of this one must be removed before this item can be removed. For appli- 
cation evaluations, thè elements on thè stack are expressions to evaluate. To 
finish evaluating thè first expression, all of its component subexpressions must 
be evaluated. Hence, thè first application evaluation started is thè last one to 
finish. 

Exercise 4.16. This exercise tests your understanding of thè ( factorial 2) evalua- 
tion. 

a. In step 5, thè second part of thè application evaluation rule, Rule 3(b), is used. 
In which step does this evaluation rule complete? 

b. In step 11, thè first part of thè application evaluation rule, Rule 3(a), is used. 
In which step is thè following use of Rule 3(b) started? 

c. In step 25, thè first part of thè application evaluation rule, Rule 3(a), is used. 
In which step is thè following use of Rule 3(b) started? 

d. To evaluate ( factorial 3), how many times would Evaluation Rule 2 be used to 
evaluate thè name factoriaU 

e. [*] To evaluate ( factorial ri) for any positive integer n, how many times would 
Evaluation Rule 2 be used to evaluate thè name factoriaU 

Exercise 4.17. For which input values n will an evaluation of ( factorial ri) even- 
tually reach a value? For values where thè evaluation is guaranteed to finish, 
make a convincing argument why it must finish. For values where thè evalua- 
tion would not finish, explain why. 


4.5 Developing Complex Programs 

To develop and use more complex procedures it will be useful to learn some 
helpful techniques for understanding what is going on when procedures are 
evaluated. It is very rare for a first version of a program to be completely correct, 
even for an expert programmer. Wise programmers build programs incremen- 
tally, by writing and testing small components one at a time. 

The process of fixing broken programs is known as debugging. The key to de- debugging 
bugging effectively is to be systematic and thoughtful. It is a good idea to take 
notes to keep track of what you have learned and what you have tried. Thought- 
less debugging can be very frustrating, and is unlikely to lead to a correct pro- 
gram. 

A good strategy for debugging is to: 
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Ensure you understand thè intended behavior of your procedure. Think of 
a few representative inputs, and what thè expected output should be. 

Do experiments to observe thè actual behavior of your procedure. Try your 
program on simple inputs first. What is thè relationship between thè ac- 
tual outputs and thè desired outputs? Does it work correctly for some in- 
puts but not others? 

Make changes to your procedure and retest it. If you are not sure what to 
do, make changes in small steps and carefully observe thè impact of each 
change. 


First actual bug 

Grace Hopper’s notebook, 1947 


For more complex programs, follow this strategy at thè level of sub-components. 
For example, you can try debugging at thè level of one expression before trying 
thè whole procedure. Break your program into several procedures so you can 
test and debug each procedure independently. The smaller thè unit you test at 
one time, thè easier it is to understand and fìx problems. 


DrRacket provides many useful and powerful features to aid debugging, but thè 
most important tool for debugging is using your brain to think carefully about 
what your program should be doing and how its observed behavior differs from 
thè desired behavior. Next, we describe two simple ways to observe program 
behavior. 


4.5.1 Printing 

One useful procedure built-in to DrRacket is thè display procedure. It takes one 
input, and produces no output. Instead of producing an output, it prints out thè 
value of thè input (it will appear in purple in thè Interactions window). We can 
use display to observe what a procedure is doing as it is evaluated. 

For example, if we add a ( display n) expression at thè beginning of our factorial 
procedure we can see all thè intermediate calls. To make each printed value 
appear on a separate line, we use thè newline procedure. The newline procedure 
prints a new line; it takes no inputs and produces no output. 

(define ( factorial ri) 

( display "Enter factorial: ") {display n) ( newline ) 

(if (— n 0) 1 (* n {factorial (— n 1))))) 

Evaluating ( factorial 2) produces: 


Enter factorial: 2 
Enter factorial: 1 
Enter factorial: 0 
2 


The built-in printf procedure makes it easier to print out many values at once. 
It takes one or more inputs. The first input is a string (a sequence of characters 
enclosed in doublé quotes). The string can include special ~a markers that print 
out values of objects inside thè string. Each ~a marker is matched with a corre- 
sponding input, and thè value of that input is printed in place of thè ~a in thè 
string. Another special marker, ~n, prints out a new line inside thè string. 

Using printf, we can define our factorial procedure with printing as: 


Chapter 4. Problems and Procedures 


69 


(define ( factorial ri) 

( printf "Enter factorial: ~a~n" n ) 

(if (— n 0) 1 (* n ( factorial {— n 1))))) 

The display, printf, and newline procedures do not produce output values. In- 
stead, they are applied to produce side effects. A side effect is something that side effects 
changes thè state of a computation. In this case, thè side effect is printing in 
thè Interactions window. Side effects make reasoning about what programs do 
much more complicated since thè order in which events happen now matters. 

We will mostly avoid using procedures with side effects until Chapter 9, but 
printing procedures are so useful that we introduce thern here. 

4.5.2 Tracing 

DrRacket provides a more automated way to observe applications of procedures. 

We can use tracing to observe thè start of a procedure evaluation (including thè 
procedure inputs) and thè completion of thè evaluation (including thè output). 

To use tracing, it is necessary to first load thè tracing library by evaluating this 
expression: 

( require racket/trace) 

This defines thè trace procedure that takes one input, a constructed procedure 
(trace does not work for primitive procedures). After evaluating ( trace proc), thè 
interpreter will print out thè procedure name and its inputs at thè beginning 
of every application of proc and thè value of thè output at thè end of thè ap- 
plication evaluation. If there are other applications before thè first application 
finishes evaluating, these will be printed indented so it is possible to match up 
thè beginning and end of each application evaluation. For example (thè trace 
outputs are shown in typewriter font), 

> {trace factorial) 

> ( factorial 2) 

(factorial 2) 

I (factorial 1) 

I (factorial 0) 

1 1 
11 

2 
2 

The trace shows that ( factorial 2) is evaluated first; within its evaluation, {facto- 
rial 1 ) and then {factorial 0) are evaluated. The outputs of each of these appli- 
cations is lined up vertically below thè application entry trace. 


Exploration 4.2: Recipes for n 


The value n is thè defined as thè ratio between thè circumference of a circle and 
its diameter. One way to calculate thè approximate value of n is thè Gregory- 
Leibniz series (which was actually discovered by thè Indian mathematician Màd- 
hava in thè 14 J?1 century): 
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This summation converges to n. The more terms that are included, thè closer 
thè computed value will be to thè actual value of n. 

a. [*] Defitte a procedure compute-pi that takes as input n, thè number of terms 
to include and outputs an approximation of n computed using thè first n 
terms of thè Gregory- Leibniz series. ( compute-pi 1) should evaluate to 4 and 
( compute-pi 2) should evaluate to 2 2/3. For higher terms, use thè built-in 
procedure exact->inexact to see thè decimai value. For example, 

( exact->inexact ( compute-pi 10000)) 
evaluates (after a long wait!) to 3.1414926535900434. 


The Gregory-Leibniz series is fairly simple, but it takes an awful long time to con- 
verge to a good approximation for n — only one digit is correct after 10 terms, 
and after summing 10000 terms only thè first four digits are correct. 

Màdhava discovered another series for computing thè value of n that converges 
much more quickly: 
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3*3 5 * 3 2 7 * 3 3 9 * 3 4 


Madhava computed thè first 21 terms of this series, finding an approximation of 
7r that is correct for thè first 12 digits: 3.14159265359. 

b. [**] Define a procedure cherry-pi that takes as input n, thè number of terms 
to include and outputs an approximation of 7r computed using thè first n 
terms of thè Màdhava series. (Continue reading for hints.) 

To defin efaster-pi, first define two helper functions: faster-pi-helper, that takes 
one input, n, and computes thè sum of thè first n terms in thè series without thè 
\/T2 factor, and faster-pi-term that takes one input n and computes thè value 
of thè n th term in thè series (without alternating thè adding and subtracting). 

( faster-pi-term 1) should evaluate to 1 and ( faster-pi-term 2) should evaluate to 
1/9. Then, defin efaster-pi as: 

(define ( faster-pi terms) (* ( sqrt 12) ( faster-pi-helper terms))) 

This uses thè built-in sqrt procedure that takes one input and produces as out- 
put an approximation of its square root. The accuracy of thè sqrt procedure 7 
limits thè number of digits of n that can be correctly computed using this method 
(see Exploration 4.1 for ways to compute a more accurate approximation for thè 
square root of 12). You should be ab le to get a few more correct digits than 
Màdhava was able to get without a computer 600 years ago, but to get more 
digits would need a more accurate sqrt procedure or another method for com- 
puting 7T. 

The built-in expt procedure takes two inputs, a and b, and produces a b as its 
output. You could also define your own procedure to compute a b for any integer 
inputs a and b. 

c. [* * *] Find a procedure for computing enough digits of n to find thè Feyn- 
man point where there are six consecutive 9 digits. This point is named for 
Richard Feynman, who quipped that he wanted to memorize n to that point 
so he could recite it as “. . . nine, nine, nine, nine, nine, nine, and so on”. 


7 To test its accuracy, try evaluating [square ( sqrt 12)). 
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Exploration 4.3: Recursive Definitions and Games 


Many games can be analyzed by thinking recursively. For this exploration, we 
consider how to develop a winning strategy for some two-player games. In all 
thè games, we assume player 1 moves first, and thè two players take turns until 
thè game ends. The game ends when thè player who’s turn it is cannot move; thè 
other player wins. A strategy is a winning strategy if it provides a way to always 
select a move that wins thè game, regardless of what thè other player does. 

One approach for developing a winning strategy is to work backwards from thè 
winning position. This position corresponds to thè base case in a recursive def- 
inition. If thè game reaches a winning position for player 1, then player 1 wins. 
Moving back one move, if thè game reaches a position where it is player 2’s move, 
but all possible moves lead to a winning position for player 1, then player 1 is 
guaranteed to win. Continuing backwards, if thè game reaches a position where 
it is player l’s move, and there is a move that leads to a position where all pos- 
sible moves for player 2 lead to a winning position for player 1, then player 1 is 
guaranteed to win. 

The first game we will consider is called Nini. Variants on Nim have been played 
widely over many centuries, but no one is quite sure where thè name comes 
from. We’ll start with a simple variation on thè game that was called Thai 21 
when it was used as an Immunity Challenge on Survivor. 

In this version of Nim, thè game starts with a pile of 21 stones. One each turn, a 
player removes one, two, or three stones. The player who removes thè last stone 
wins, since thè other player cannot make a valid move on thè following turn. 

a. What should thè player who moves first do to ensure she can always win thè 
game? (Hint: start with thè base case, and work backwards. Think about a 
game starting with 5 stones first, before trying 21.) 

b. Suppose instead of being able to take 1 to 3 stones with each turn, you can 
take 1 to n stones where n is some number greater than or equal to 1. For 
what values of n should thè first player always win (when thè game starts 
with 21 stones)? 

A standard Nim game starts with three heaps. At each turn, a player removes any 
number of stones from any one heap (but may not remove stones from more 
than one heap). We can describe thè state of a 3-heap game of Nim using three 
numbers, representing thè number of stones in each heap. For example, thè 
Thai 21 game starts with thè state (21 0 0) (one heap with 21 stones, and two 
empty heaps). 8 

c. What should thè first player do to win if thè starting state is (2 1 0)? 

d. Which player should win if thè starting state is (2 2 2)? 

e. [*] Which player should win if thè starting state is (5 6 7)? 

f. [**] Describe a strategy for always winning a winnable game of Nim starting 
from any position. 9 

8 With thè standard Nim rules, this would not be an interesting game since thè first player can 
simply win by removing all 21 stones from thè first heap. 

9 lf you get stuck, you’ll fìnd many resources about Nim on thè Internet; but, you’ll get a lot more 
out of this if you solve it yourself. 
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The final game we consider is thè “Corner thè Queen” game invented by Rufus 
Isaacs. 10 The game is played using a single Queen on a arbitrarily large chess- 
board as shown in Figure 4.5. 
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Figure 4.5. Cornering thè Queen. 


On each turn, a player moves thè Queen one or more squares in either thè left, 
down, or diagonally down-left direction (unlike a standard chess Queen, in this 
game thè queen may not move right, up or up-right). As with thè other games, 
thè last player to make a legai move wins. For this game, once thè Queen reaches 
thè bottom left square marked with thè *, there are no moves possible. Hence, 
thè player who moves thè Queen onto thè * wins thè game. We name thè squares 
using thè numbers on thè sides of thè chessboard with thè column number first. 
So, thè Queen in thè picture is on square (4 7). 

g. Identify all thè starting squares for which thè first played to move can win 
right away. (Your answer should generalize to any size square chessboard.) 

h. Suppose thè Queen is on square (2 1) and it is your move. Explain why there 
is no way you can avoid losing thè game. 

i. Given thè shown starting position (with thè Queen at (4 7), would you rather 
be thè first or second player? 

j. [*] Describe a strategy for winning thè game (when possible). Explain from 
which starting positions it is not possible to win (assuming thè other player 
always makes thè right move). 

k. [*] Define a variant of Nim that is essentially thè same as thè “Corner thè 
Queen” game. (This game is known as “Wythoff’s Nim”.) 

Developing winning strategies for these types of games is similar to defining a 
recursive procedure that solves a problem. We need to identify a base case from 
which it is obvious how to win, and a way to make progress fio m a large input 
towards that base case. 

10 Described in Martin Gardner, Penrose Tiles to Trapdoor Ciphers. . .And thè Return ofDr Matrix, 
The Mathematica! Association of America, 1997. 
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4.6 Summary 

By breaking problems down into simpler problems we can develop Solutions 
to complex problems. Many problems can be solved by combining instances 
of thè same problem on simpler inputs. When we define a procedure to solve a 
problem this way, it needs to have a predicate expression to determine when thè 
base case has been reached, a consequent expression that provides thè value for 
thè base case, and an alternate expression that defines thè solution to thè given 
input as an expression using a solution to a smaller input. 

Our generai recursive problem solving strategy is: 

1. Be optimistic! Assume you can solve it. 

2. Think of thè simplest version of thè problem, something you can already 
solve. This is thè base case. 

3. Consider how you would solve a big version of thè problem by using thè 
result for a slightly smaller version of thè problem. This is thè recursive 
case. 


4. Combine thè base case and thè recursive case to solve thè problem. 

For problems involving numbers, thè base case is often when thè input value 
is zero. The problem size is usually reduced is by subtracting 1 from one of thè 
inputs. 


In thè next chapter, we introduce more complex data structures. For problems 
involving complex data, thè same strategy will work but with different base cases 
and ways to shrink thè problem size. 


TOUR. OF ACCOUNTING 

OVER HERE 
WE HAVE OUR 
RANDOIA NUIABER 
GENERATOR. 





I’d rather be an 
optimist and afool 
than a pessimist 
and right. 

Albert Einstein 
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online library 



Data 


From a bit to a few hundred megabytes, from a microsecond to halfan hour of computing 
conp-onts us with thè completely baffling ratio of IO 9 ! .... Byevokingthe needfor deep 
conceptual hierarchies, thè automatic computer confionts us with a radically new 
intellectual challenge that has no precedent in our history. 

Edsger Dijkstra 


For all thè programs so far, we have been limited to simple data such as numbers 
and Booleans. We cali this scalar data since it has no structure. As we saw in scalar 
Chapter 1, we can represent all discrete data using just (enormously large) whole 
numbers. For example, we could represent thè text of a book using only one 
(very large!) number, and manipulate thè characters in thè book by changing thè 
value of that number. But, it would be very difficult to design and understand 
computations that use numbers to represent complex data. 

We need more complex data structures to better model structured data. We want 
to represent data in ways that allow us to think about thè problem we are trying 
to solve, rather than thè details of how data is represented and manipulated. 

This chapter covers techniques for building data structures and for defming pro- 
cedures that manipulate structured data, and introduces data abstraction as a 
tool for managing program complexity. 

5.1 Types 

All data in a program has an associated type. Internally, all data is stored just 
as a sequence of bits, so thè type of thè data is important to understand what it 
means. We have seen several different types of data already: Numbers, Booleans, 
and Procedures (we use initial capitai letters to signify a datatype). 

A datatype defines a set (often infinite) of possible values. The Boolean datatype datatype 
contains thè two Boolean values, true and false. The Number type includes thè 
infinite set of all whole numbers (it also includes negative numbers and rational 
numbers). We think of thè set of possible Numbers as infinite, even though on 
any particular computer there is some limit to thè amount of memory available, 
and hence, some largest number that can be represented. On any reai com- 
puter, thè number of possible values of any data type is always finite. But, we 
can imagine a computer large enough to represent any given number. 

The type of a value determines what can be done with it. For example, a Number 
can be used as one of thè inputs to thè primitive procedures +, *, and =. A 
Boolean can be used as thè first subexpression of an if expression and as thè 
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input to thè not procedure ( — not — can also take a Number as its input, but for 
all Number value inputs thè output is false), but cannot be used as thè input to 
+, *, or —} 

A Procedure can be thè first subexpression in an application expression. There 
are inbnitely many different types of Procedures, since thè type of a Procedure 
depends on its input and output types. For example, recali bigger procedure 
from Chapter 3: 

(define ( bigger a b) (if (> a b) a b)) 

It takes two Numbers as input and produces a Number as output. We denote 
this type as: 


Number x Number — » Number 


The inputs to thè procedure are shown on thè left side of thè arrow. The type of 
each input is shown in order, separated by thè x Symbol. 1 2 The output type is 
given on thè right side of thè arrow. 

From its definition, it is clear that thè bigger procedure takes two inputs from its 
parameter list. How do we know thè inputs must be Numbers and thè output is 
a Number? 

The body of thè bigger procedure is an if expression with thè predicate expres- 
sion (> ab). This applies thè > primitive procedure to thè two inputs. The 
type of thè > procedure is Number x Number — »■ Boolean. So, for thè predi- 
cate expression to be valid, its inputs must both be Numbers. This means thè 
input values to bigger must both be Numbers. We know thè output of thè bigger 
procedure will be a Number by analyzing thè consequent and alternate subex- 
pressions: each evaluates to one of thè input values, which must be a Number. 

Starting with thè primitive Boolean, Number, and Procedure types, we can build 
arbitrarily complex datatypes. This chapter introduces mechanisms for building 
complex datatypes by combining thè primitive datatypes. 


Exercise 5.1. Describe thè type of each of these expressions. 

a. 17 

b. (lambda (, a ) (> a 0)) 

c. ((lambda {a) (> a 0)) 3) 

d. (lambda ( a ) (lambda (b) (> a b))) 

e. (lambda ( a ) a) 


1 The primitive procedure equal? is a more generai comparison procedure that can take as in- 
puts any two values, so could be used to compare Boolean values. For example, ( equal ? false false) 
evaluates to true and ( equal ? trae 3) is a valid expression that evaluates to false. 

2 The notation using x to separate input types makes sense if you think about thè number of 
different inputs to a procedure. For example, consider a procedure that takes two Boolean values as 
inputs, so its type is Boolean x Boolean -> Value. Each Boolean input can be one of two possible 
values. If we combined both inputs into one input, there would be 2 x 2 different values needed to 
represent all possible inputs. 
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Exercise 5.2. Define or identify a procedure that has thè given type. 

a. Number x Number — > Boolean 

b. Number — >• Number 

c. (Number — > Number) x (Number — > Number) 

— > (Number — > Number) 

d. Number — > (Number — > (Number — » Number)) 


5.2 Pairs 

The simplest structured data construct is a Pair. We draw a Pair as two boxes, Pair 
each containing a value. We cali each box of a Pair a celi. Here is a Pair where thè 
first celi has thè value 37 and thè second celi has thè value 42: 


37 


42 


Scheme provides built-in procedures for constructing a Pair, and for extracting 
each celi from a Pair: 


cons : Value x Value —> Pair 

Evaluates to a Pair whose first celi is thè first input and second celi is thè 
second input. The inputs can be of any type. 

car: Pair — > Value 

Evaluates to thè first celi of thè input, which must be a Pair. 
cdr : Pair — > Value 

Evaluates to thè second celi of input, which must be a Pair. 

These rather unfortunate names come from thè originai LISP implementation 
on thè IBM 704. The name cons is short for “construct”. The name car is short for 
“ Contents of thè Address part of thè Register” and thè name cdr (pronounced 
“could-er”) is short for “ Contents of thè Dee re me ni part of thè Regi s Le r”. The de- 
signer of thè originai LISP implementation picked thè names because of how 
pairs could be implemented on thè IBM 704 using a single register to store both 
parts of a pair, but it is a mistake to name things after details of their implemen- 
tation (see Section 5.6). Unfortunately, thè names stuck. 

We can construct thè Pair shown above by evaluating ( cons 37 42). DrRacket 
displays a Pair by printing thè value of each celi separated by a dot: (37 . 42). The 
interactions below show example uses of cons, car, and cdr. 

> (define mypair ( cons 37 42)) 

> ( car mypair ) 

37 

> ( cdr mypair ) 

42 

The values in thè cells of a Pair can be any type, including other Pairs. For exam- 
ple, this definition defines a Pair where each celi of thè Pair is itself a Pair: 

(define doublepair ( cons ( cons 1 2) ( cons 3 4))) 
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We can use thè car and cdr procedures to access components of thè doublepair 
structure: ( car doublepair ) evaluates to thè Pair (1 .2), and ( cdr doublepair) 
evaluates to thè Pair (3.4). 

We can compose multiple car and cdr applications to extract components from 
nested pairs: 

> ( cdr ( car doublepair )) 

2 

> [car [cdr doublepair )) 

3 

> (( fcompose cdr cdr) doublepair) /compose from Section 4.2.1 

4 

> [car [car [car doublepair))) 

Q car: expects argument of type <pair>; given 1 


The last expression produces an error when it is evaluated since car is applied 
to thè scalar value 1 . The car and cdr procedures can only be applied to an 
input that is a Pair. Hence, an error results when we attempt to apply car to 
a scalar value. This is an important property of data: thè type of data (e.g., a 
Pair) defines how it can be used (e.g., passed as thè input to car and cdr). Every 
procedure expects a certain type of inputs, and typically produces an error when 
it is applied to values of thè wrong type. 

We can draw thè value of doublepair by nesting Pairs within cells: 



Drawing Pairs within Pairs within Pairs can get quite diffìcult, however. For in- 
stance, try drawing [cons 1 [cons 2 [cons 3 ( cons 4 5)))) this way. 

Instead, we us arrows to point to thè contents of cells that are not simple values. 
This is thè structure of doublepair shown using arrows: 



Using arrows to point to celi contents allows us to draw arbitrarily complicated 
data structures such as [cons 1 [cons 2 [cons 3 [cons 4 5)))), keeping thè cells 
reasonable sizes: 
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Exercise 5.3. Suppose thè following defìnition has been executed: 

(define tpair 

( cons ( cons ( cons 1 2) ( cons 3 4)) 

5)) 

Draw thè structure defined by tpair, and give thè value of each of thè following 
expressions. 

a. ( cdr tpair ) 

b. ( car ( car ( car tpair))) 

c. ( cdr ( cdr ( car tpair))) 

d. ( car ( cdr ( cdr tpair))) 

Exercise 5.4. Write expressions that extract each of thè four elements from 
fstruct defined by (define J'sLrucL ( cons 1 ( cons 2 ( cons 3 4)))). 

Exercise 5.5. Give an expression that produces thè structure shown below. 



5.2.1 MakingPairs 

Although Scheme provides thè built-in procedures cons, car, and cdr for creat- 
ing Pairs and accessing their cells, there is nothing magical about these proce- 
dures. We can defìne procedures with thè same behavior ourselves using thè 
subset of Scheme introduced in Chapter 3. 

Here is one way to defìne thè pair procedures (we prepend an 5 to thè names to 
avoid confusion with thè built-in procedures]: 

(define ( scons a b) (lambda ( w) (if w a b) ) ) 

(define ( scar pair) ( pair true)) 

(define ( scdr pair) {pair false)) 

The scons procedure takes thè two parts of thè Pair as inputs, and produces as 
output a procedure. The output procedure takes one input, a selector that de- 
termines which of thè two cells of thè Pair to output. If thè selector is true, thè 
value of thè if expression is thè value of thè fìrst celi; if thè selector is false, it is 
thè value of thè second celi. The scar and scdr procedures apply a procedure 
constructed by scons to either true (to select thè fìrst celi in scar) or false (to 
select thè second celi in scdr). 
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Exercise 5.6. Convince yourself thè defìnitions of scons, scar, and scdr above 
work as expected by following thè evaluation rules to evaluate 

[scar ( scons 1 2)) 

Exercise 5.7. Show thè corresponding defìnitions of tcar and tcdr that provide 
thè pair selection behavior for a pair created using tcons defined as: 

(define ( tcons a b ) (lambda (w) (if w b a])) 

5.2.2 Triples to Octuples 

Pairs are useful for representing data that is composed of two parts such as a 
calendar date (composed of a number and month), or a playing card (composed 
of a rank and suit). But, what if we want to represent data composed of more 
than two parts such as a date (composed of a number, month, and year) or a 
poker hand consisting of five playing cards? For more complex data structures, 
we need data structures that have more than two components. 

A triple has three components. Here is one way to define a triple datatype: 

(define (make-triple ab c) 

(lambda (w) (if (= w 0) a (if (= w 1) b c)))) 

(define (triple-flrst L) (l 0)) 

(define ( triple-second t ) (L 1)) 

(define [triple- third t) (L 2)) 

Since a triple has three components we need three different selector values. 

Another way to make a triple would be to combine two Pairs. We do this by 
making a Pair whose second celi is itself a Pair: 

(define ( make-triple ab c) ( cons a ( cons b c))) 

(define ( triple-flrst t) ( car t)) 

(define ( triple-second t ) ( car ( cdr l))) 

(define ( triple-third t ) ( cdr ( cdr /))) 

Similarly, we can define a quadruple as a Pair whose second celi is a triple: 

(define ( make-quad ab c d) ( cons a ( make-triple b c d))) 

(define ( quad-flrst q) ( car q)) 

(define ( quad-second q) ( triple-flrst ( cdr q)) 

(define ( quad-third q) ( triple-second ( cdr q)) 

(define ( quad-fourth q) ( triple-third ( cdr q)) 

We could continue in this manner defining increasingly large tuples. 

A triple is a Pair whose second celi is a Pair. 

A quadruple is a Pair whose second celi is a triple. 

A quintuple is a Pair whose second celi is a quadruple. 

An n + 1 -uple is a Pair whose second celi is an n-uple. 

Building from thè simple Pair, we can construct tuples containing any number 
of components. 
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Exercise 5.8. Defìne a procedure that constructs a quintuple and procedures 
for selecting thè five elements of a quintuple. 

Exercise 5.9. Another way of thinking of a triple is as a Pair where thè first celi is 
a Pair and thè second celi is a scalar. Provide definitions of make-triple, triple- 
flrst, triple-second, and triple-third for this construct. 


5.3 Lists 

In thè previous section, we saw how to construct arbitrarily large tuples from 
Pairs. This way of managing data is not very satisfying since it requires defining 
different procedures for constructing and accessing elements of every length tu- 
ple. For many applications, we want to be able to manage data of any length 
such as all thè items in a web store, or all thè bids on a given item. Since thè 
number of components in these objects can change, it would be very painful to 
need to defìne a new tuple type every time an item is added. We need a data 
type that can hold any number of items. 

This definition almost provides what we need: 

An any-uple is a Pair whose second celi is an any-uple. 

This seems to allow an any-uple to contain any number of elements. The prob- 
lem is we have no stopping point. With only thè definition above, there is no 
way to construct an any-uple without already having one. 

The situation is similar to defining MoreDigits as zero or more digits in Chap- 
ter 2, defining MoreExpressions in thè Scheme grammar in Chapter 3 as zero or 
more Expressions, and recursive composition in Chapter 4. 

Recali thè grammar rules for MoreExpressions : 

MoreExpressions ::=>- Expression MoreExpressions 
MoreExpressions ::=^> e 

The rule for constructing an any-uple is analogous to thè first MoreExpression 
replacement rule. To allow an any-uple to be constructed, we also need a con- 
struction rule similar to thè second rule, where MoreExpression can be replaced 
with nothing. Since it is hard to type and read nothing in a program, Scheme 
has a name for this value: nuli. 

DrRacket will print out thè value of nuli as (). It is also known as thè empty list, 
since it represents thè List containing no elements. The built-in procedure nuli? 
takes one input parameter and evaluates to true if and only if thè value of that 
parameter is nuli. 

Using nuli, we can now define a List : 

A List is either (1) nuli or (2) a Pair whose second celi is a List. 

Symbolically, we define a List as: 

List ::=>- nuli 

List ::=> ( cons Value List ) 


nuli 


List 
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These two rules define a List as a data structure that can contain any number of 
elements. Starting from nuli, we can create Lists of any length: 

• nuli evaluates to a List containing no elements. 

• ( cons 1 nuli) evaluates to a List containing one element. 

• ( cons 1 ( cons 2 nuli)) evaluates to a List containing two elements. 

• ( cons 1 (cons 2 (cons 3 nuli))) evaluates to a 3-element List. 

Scheme provides a convenient procedure, list, for constructing a List. The list 
procedure takes zero or more inputs, and evaluates to a List containing those 
inputs in order. The following expressions are equivalent to thè corresponding 
expressions above: [list), ( list 1), {list 1 2), and ( list 1 2 3). 

Lists are just a collection of Pairs, so we can draw a List using thè same box and 
arrow notation we used to draw structures created with Pairs. Here is thè struc- 
ture resulting from {list 12 3): 


1 



There are three Pairs in thè List, thè second celi of each Pair is a List. For thè 
third Pair, thè second celi is thè List nuli, which we draw as a slash through thè 
final celi in thè diagram. 

Table 5.1 summarizes some of thè built-in procedures for manipulating Pairs 
and Lists. 

Exercise 5.10. For each of thè following expressions, explain whether or not thè 
expression evaluates to a List. Check your answers with a Scheme interpreter by 
using thè list? procedure. 

a. nuli 

b. ( cons 1 2) 

c. ( cons nuli nuli) 

d. ( cons ( cons ( cons 1 2) 3) nuli) 

e. ( cdr { cons 1 ( cons 2 ( cons nuli nuli)))) 

f. {cons {list 1 2 3) 4) 



Type 

Output 

cons 

Value x Value — > Pair 

a Pair consisting of thè two inputs 

car 

Pair — > Value 

thè fìrst celi of thè input Pair 

cdr 

Pair — > Value 

thè second celi of thè input Pair 

list 

zero or more Values — > List 

a List containing thè inputs 

nuli? 

Value — > Boolean 

true if thè input is nuli, otherwise false 

pair? 

Value — > Boolean 

true if thè input is a Pair, otherwise false 

list? 

Value — > Boolean 

true if thè input is a List, otherwise false 


Table 5.1. Selected Built-In Scheme Procedures for Lists and Pairs. 
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5.4 List Procedures 

Since thè List data structure is defined recursively, it is naturai to define recur- 
sive procedures to examine and manipulate lists. Whereas most recursive pro- 
cedures on inputs that are Numbers usually used 0 as thè base case, for lists thè 
most common base case is nuli. With numbers, we make progress by subtract- 
ing 1 ; with lists, we make progress by using cdr to reduce thè length of thè input 
List by one element for each recursive application. This means we often break 
problems involving Lists into figuring out what to do with thè first element of 
thè List and thè result of applying thè recursive procedure to thè rest of thè List. 

We can specialize our generai problem solving strategy from Chapter 3 for pro- 
cedures involving lists: 

1. Be very optimistic! Since lists themselves are recursive data structures, 
most problems involving lists can be solved with recursive procedures. 

2. Think of thè simplest version of thè problem, something you can already 
solve. This is thè base case. For lists, this is usually thè empty list. 

3. Consider how you would solve a big version of thè problem by using thè 
result for a slightly smaller version of thè problem. This is thè recursive 
case. For lists, thè smaller version of thè problem is usually thè rest [cdr) 
of thè List. 

4. Combine thè base case and thè recursive case to solve thè problem. 

Next we consider procedures that examine lists by walking through their ele- 
ments and producing a scalar value. Section 5.4.2 generalizes these procedures. 
In Section 5.4.3, we explore procedures that output lists. 

5.4. 1 Procedures that Examine Lists 

All of thè example procedures in this section take a single List as input and pro- 
duce a scalar value that depends on thè elements of thè List as output. These 
procedures have base cases where thè List is empty, and recursive cases that ap- 
ply thè recursive procedure to thè cdr of thè input List. 


Example 5.1: Length 


How many elements are in a given List? 3 Our standard recursive problem solv- 
ing technique is to “Think of thè simplest version of thè problem, something 
you can already solve.” For this procedure, thè simplest version of thè problem 
is when thè input is thè empty list, nuli. We know thè length of thè empty list is 
0. So, thè base case test is [nuli? p) and thè output for thè base case is 0. 

For thè recursive case, we need to consider thè structure of all lists other than 
nuli. Recali from our definition that a List is either nuli or ( cons Value List). The 
base case handles thè nuli list; thè recursive case must handle a List that is a Pair 
of an element and a List. The length of this List is one more than thè length of 
thè List that is thè cdr of thè Pair. 

3 Scheme provides a built-in procedure length that takes a List as its input and outputs thè num- 
ber of elements in thè List. Here, we will define our own list-length procedure that does this (without 
using thè built-in length procedure). As with many other examples and exercises in this chapter, it 
is instructive to define our own versions of some of thè built-in list procedures. 
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(define ( list-length p) 

(if ( nuli ? p) 

0 

(+ 1 ( list-length ( cdr p))))) 

Here are a few example applications of our list-length procedure: 

> ( list-length nuli) 

0 

> ( list-length {cons 0 nuli)) 

1 

> ( list-length [lisi 1 2 3 4)) 

4 


Example 5.2: List Sums and Products 


First, we define a procedure that takes a List of numbers as input and produces 
as output thè sum of thè numbers in thè input List. As usuai, thè base case is 
when thè input is nuli : thè sum of an empty list is 0. For thè recursive case, we 
need to add thè value of thè first number in thè List, to thè sum of thè rest of thè 
numbers in thè List. 

(define (lisi- sum p) 

(if [nuli? p) 0 (+ (car p) [list- sum ( cdr p))))) 

We can define list-product similarly, using * in place of +. The base case re- 
sult cannot be 0, though, since then thè final result would always be 0 since any 
number multiplied by 0 is 0. We follow thè mathematical convention that thè 
product of thè empty list is 1 . 

(define ( list-product p) 

(if ( nuli ? p) 1 (* (car p) ( list-product ( cdr p))))) 


Exercise 5.1 1. Define a procedure is-list? that takes one input and outputs true if 
thè input is a List, and false otherwise. Your procedure should behave identically 
to thè built-in list? procedure, but you should not use list? in your definition. 

Exercise 5.12. Define a procedure list-max that takes a List of non-negative 
numbers as its input and produces as its result thè value of thè greatest element 
in thè List (or 0 if there are no elements in thè input List). For example, ( list-max 
(, list 112 0)) should evaluate to 2. 

5.4.2 Generic Accumulators 

The list-length, list-sum, and list-product procedures all have very similar struc- 
tures. The base case is when thè input is thè empty list, and thè recursive case 
involves doing something with thè first element of thè List and recursively call- 
ing thè procedure with thè rest of thè List: 

(define ( Recursive-Procedure p) 

(if ( nuli ? p) 

Base-Case-Result 

( Accumulator-Function ( car p) ( Recursive-Procedure ( cdr p))))) 

We can define a generic accumulator procedure for lists by making thè base case 
result and accumulator function inputs: 
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(define (list- accumulate f base p) 

(if ( nuli ? p) 
base 

(f ( car p) (lisi -acci t ni u late J' base (cdr /?))))) 

We can use list- accumulate to define list-sum and list-product : 

(define ( list-sum p) ( list-accumulate + 0 p)) 

(define ( list-product p) ( list-accumulate * 1 p)) 

Defining thè list-length procedure is a bit less naturai. The recursive case in thè 
originai list-length procedure is (+ 1 ( list-length ( cdr p))); it does not use thè 
value of thè first element of thè List. But, list-accumulate is defined to take a 
procedure that takes two inputs — thè first input is thè first element of thè List; 
thè second input is thè result of applying list-accumulate to thè rest of thè List. 
We should follow our usuai strategy: be optimistic! Being optimistic as in recur- 
sive definitions, thè value of thè second input should be thè length of thè rest of 
thè List. Hence, we need to pass in a procedure that takes two inputs, ignores 
thè first input, and outputs one more than thè value of thè second input: 

(define ( list-length p) 

( list-accumulate (lambda ( el length-rest) (+ 1 length-rest )) 0 p)) 


Exercise 5.13. Use list-accumulate to define list-max (from Exercise 5.12). 
Exercise 5.14. [*] Use list-accumulate to define is-list? (from Exercise 5.11). 


Example 5.3: Accessing List Elements 


The built-in car procedure provides a way to get thè first element of a list, but 
what if we want to get thè third element? We can do this by taking thè cdr twice 
to eliminate thè first two elements, and then using car to get thè third: 

(car (cdr (cdr p))) 

We want a more generai procedure that can access any selected list element. It 
takes two inputs: a List, and an index Number that identifies thè element. If we 
start counting from 1 (it is often more naturai to start from 0), then thè base case 
is when thè index is 1 and thè output should be thè first element of thè List: 

(if (— n 1) (car p) . . .) 

For thè recursive case, we make progress by eliminating thè first element of thè 
list. We also need to adjust thè index: since we have removed thè first element 
of thè list, thè index should be reduced by one. For example, instead of wanting 
thè third element of thè originai list, we now want thè second element of thè cdr 
of thè originai list. 

(define (list-get-element p n ) 

(if (— n 1) 

(car p) 

(list-get-element (cdr p) (- n 1)))) 

What happens if we apply list-get-element to an index that is larger than thè size 
of thè input List (for example, (list-get-element (list 1 2) 3))? 
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The first recursive cali is ( list-get-element (lisi 2) 2). The second recursive cali is 
( list-get-element (lisi) 1 ). At this point, n is 1 , so thè base case is reached and (car 
p) is evaluated. But, p is thè empty list (which is not a Pair), so an error results. 

A better version of list-get-element would provide a meaningful error message 
when thè requested element is out of range. We do this by adding an if expres- 
sion that tests if thè input List is nuli. 

(define ( list-get-element p n) 

(if ( nuli ? p) 

( error "Index out of range") 

(if (= n 1) ( car p) ( list-get-element ( cdr p) (— n 1))))) 

The built-in procedure error takes a String as input. The String datatype is a 
sequence of characters; we can create a String by surrounding characters with 
doublé quotes, as in thè example. The error procedure terminates program ex- 
ecution with a message that displays thè input value. 

defensive Checking explicitly for invalid inputs is known as defensive programming. Pro- 
programming gramming defensively helps avoid tricky to debug errors and makes it easier to 
understand what went wrong if there is an error. 


Exercise 5.15. Defìne a procedure list-last- element that takes as input a List 
and outputs thè last element of thè input List. If thè input List is empty, list-last- 
element should produce an error. 

Exercise 5.16. Defìne a procedure list-ordered? that takes two inputs, a test 
procedure and a List. It outputs true if all thè elements of thè List are ordered 
according to thè test procedure. For example, (list-ordered? < ( list 1 2 3)) evalu- 
ates to true, and (list-ordered? < ( list 1 2 3 2)) evaluates to false. Hint: think about 
what thè output should be for thè empty list. 

5.4.3 Procedures that Construct Lists 

The procedures in this section take values (including Lists) as input, and pro- 
duce a new List as output. As before, thè empty list is typically thè base case. 
Since we are producing a List as output, thè result for thè base case is also usu- 
ally nuli. The recursive case will use cons to construct a List combining thè first 
element with thè result of thè recursive application on thè rest of thè List. 


Example 5.4: Mapping 


One common task for manipulating a List is to produce a new List that is thè re- 
sult of applying some procedure to every element in thè input List. 

For thè base case, applying any procedure to every element of thè empty list 
produces thè empty list. For thè recursive case, we use cons to construct a List. 
The first element is thè result of applying thè mapping procedure to thè first 
element of thè input List. The rest of thè output List is thè result of recursively 
mapping thè rest of thè input List. 

Here is a procedure that constructs a List that contains thè square of every ele- 
ment of thè input List: 
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(define ( list-square p) 

(if ( nuli ? p) nuli 

(i cons ( square ( car p)) 

( list-square ( cdr p))))) 

We generalize this by making thè procedure which is applied to each element an 
input. The procedure list-map takes a procedure as its first input and a List as 
its second input. It outputs a List whose elements are thè results of applying thè 
input procedure to each element of thè input List. 4 

(define [list-map f p) 

(if ( nuli ? p) nuli 
[cons [f [car p)) 

[list-map f [cdr p))))) 

We can use list-map to define square-all : 

(define [square-all p) [list-map square p)) 


Exercise 5.17. Define a procedure list-increment that takes as input a List of 
numbers, and produces as output a List containing each element in thè input 
List incremented by one. For example, [list-increment 1 2 3) evaluates to (2 3 4). 


Exercise 5.18. Use list-map and list-sum to define list-length : 


(define [list-length p) [list-sum [list-map 

p))) 

Example 5.5: Filtering 




Consider defining a procedure that takes as input a List of numbers, and eval- 
uates to a List of all thè non-negative numbers in thè input. For example, [list- 
filter- negative [list 1 -3-4 5 -2 0)) evaluates to (1 5 0). 


First, consider thè base case when thè input is thè empty list. If we filter thè 
negative numbers from thè empty list, thè result is an empty list. So, for thè base 
case, thè result should be nuli. 

In thè recursive case, we need to determine whether or not thè first element 
should be included in thè output. If it should be included, we construct a new 
List consisting of thè first element followed by thè result of filtering thè remain- 
ing elements in thè List. If it should not be included, we skip thè first element 
and thè result is thè result of filtering thè remaining elements in thè List. 

(define [list-filter-negative p) 

(if [nuli? p) nuli 
(if (>= [car p) 0) 

[cons [car p) [list-filter-negative [cdr p))) 

[list-filter-negative [cdr p ) ) ) ) ) 

Similarly to list-map, we can generalize our filter by making thè test procedure 
as an input, so we can use any predicate to determine which elements to include 


4 Scheme provides a built-in map procedure. It behaves like this one when passed a procedure 
and a single List as inputs, but can also work on more than one List input at a time. 
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in thè output List. 5 

(define (Usi- filler test p) 

(if ( nuli ? p ) nuli 
(if (test (car p)) 

(cons (car p) (list-fllter test (cdr p ) ) ) 

(list-fllter test (cdr /?))))) 

Using thè list-fllter procedure, we can define list-fllter-negative as: 

(define (list-fllter-negative p) (list-fllter (lambda (x) (>= xO)) p)) 

We could also define thè list-fllter procedure using thè list-accumulate proce- 
dure from Section 5.4.1: 

(define (list-fllter test p) 

(list-accumulate 

(lambda (el rest) (if (test el ) (cons el rest ) rest)) 

nuli 

p)) 

Exercise 5.19. Define a procedure list-fllter-even that takes as input a List of 
numbers and produces as output a List consisting of all thè even elements of 
thè input List. 

Exercise 5.20. Define a procedure list-remove that takes two inputs: a test pro- 
cedure and a List. As output, it produces a List that is a copy of thè input List 
with all of thè elements for which thè test procedure evaluates to true removed. 
For example, (list-remove (lambda (x) (= x 0)) (list 0 12 3)) should evaluates to 
thè List (1 2 3). 

Exercise 5.21. [**’ Define a procedure list-unique-elements that takes as input 
a List and produces as output a List containing thè unique elements of thè input 
List. The output List should contain thè elements in thè same order as thè input 
List, but should only contain thè first appearance of each value in thè input List. 


Example 5.6: Append 


The list-append procedure takes as input two lists and produces as output a List 
consisting of thè elements of thè first List followed by thè elements of thè sec- 
ond List. 6 For thè base case, when thè first List is empty, thè result of appending 
thè lists should just be thè second List. When thè first List is non-empty, we can 
produce thè result by cons- ing thè first element of thè first List with thè result of 
appending thè rest of thè first List and thè second List. 

(define (list-append p q) 

(if (nuli? p) q 

(cons (car p) (list-append (cdr p) q) )) ) 


5 Scheme provides a built-in function/zter that behaves like our list-fllter procedure. 

6 There is a built-in procedure append that does this. The built-in append takes any number of 
Lists as inputs, and appends them all into one List. 
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Example 5.7: Reverse 


The list-reverse procedure takes a List as input and produces as output a List 
containing thè elements of thè input List in reverse order. 7 For example, ( list- 
reverse ( list 1 2 3)) evaluates to thè List (3 2 1). As usuai, we consider thè base 
case where thè input List is nuli first. The reverse of thè empty list is thè empty 
list. To reverse a non-empty List, we should put thè first element of thè List at 
thè end of thè result of reversing thè rest of thè List. 

The tricky part is putting thè first element at thè end, since cons only puts ele- 
ments at thè beginning of a List. We can use thè list-append procedure defined 
in thè previous example to put a List at thè end of another List. To make this 
work, we need to turn thè element at thè front of thè List into a List containing 
just that element. We do this using {list ( car p) ) . 

(define ( list-reverse p) 

(if ( nuli ? p ) nuli 

{list-append ( list-reverse ( cdr p)) ( list ( car p))))) 

Exercise 5.22. Define thè list-reverse procedure using list- accumulate. 


Example 5.8: Intsto 


For our final example, we define thè intsto procedure that constructs a List con- 
taining thè whole numbers between 1 and thè input parameter value. For exam- 
ple, ( intsto 5) evaluates to thè List (1 2 3 4 5). 

This example combines ideas from thè previous chapter on creating recursive 
definitions for problems involving numbers, and from this chapter on lists. Since 
thè input parameter is not a List, thè base case is not thè usuai list base case 
when thè input is nuli. Instead, we use thè input value 0 as thè base case. The 
result for input 0 is thè empty list. For higher values, thè output is thè result of 
putting thè input value at thè end of thè List of numbers up to thè input value 
minus one. 

A first attempt that doesn’t quite work is: 

(define ( revintsto ri) 

(if (= n 0) nuli 

( cons n ( revintsto (— n 1))))) 

The problem with this solution is that it is cons- ing thè higher number to thè 
front of thè result, instead of at thè end. Hence, it produces thè List of numbers 
in descending order: ( revintsto 5) evaluates to (5 4 3 2 1). 

One solution is to reverse thè result by composing list-reverse with revintsto : 

(define ( intsto ri) ( list-reverse {revintsto ri))) 

Equivalently, we can use th e /compose procedure from Section 4.2: 

(define intsto (/ compose list-reverse revintsto)) 

Alternatively, we could use list-append to put thè high number directly at thè 
end of thè List. Since thè second operand to list-append must be a List, we use 
{list ri) to make a singleton List containing thè value as we did for list-reverse. 


7 The built-in procedure reverse does this. 
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(define ( intsto ri) 

(if (= n 0) nuli 

( list-append ( intsto (—ni)) (lisi ri)))) 

Although all of these procedures are functionally equivalent (for all valid inputs, 
each function produces exactly thè same output), thè amount of computing 
work (and hence thè time they take to execute) varies across thè implemen- 
tations. We consider thè problem of estimating thè running-times of different 
procedures in Part IL 


Exercise 5.23. Define facinrial using intsto. 


5.5 ListsofLists 

The elements of a List can be any datatype, including, of course, other Lists. In 
defining procedures that operate on Lists of Lists, we often use more than one 
recursive cali when we need to go inside thè inner Lists. 


Example5.9: Summing Nested Lists 


Consider thè problem of summing all thè numbers in a List of Lists. For exam- 
ple, ( nested-list-sum ( list ( list 1 2 3) ( list 4 5 6))) should evaluate to 21. We can 
define nested-list-sum using list-sum on each List. 

(define ( nested-list-sum p) 

(if ( nuli ? p) 0 

(+ ( list-sum ( car p)) 

( nested-list-sum { cdr p))))) 

This works when we know thè input is a List of Lists. But, what if thè input can 
contain arbitrarily deeply nested Lists? 

To handle this, we need to recursively sum thè inner Lists. Each element in our 
deep List is either a List or a Number. If it is a List, we should add thè value of 
thè sum of all elements in thè List to thè result for thè rest of thè List. If it is a 
Number, we should just add thè value of thè Number to thè result for thè rest of 
thè List. So, our procedure involves two recursive calls: one for thè first element 
in thè List when it is a List, and thè other for thè rest of thè List. 

(define ( deep-list-sum p) 

(if ( nuli ? p) 0 

(+ (if {list? ( car p)) 

( deep-list-sum ( car p)) 

{car p)) 

{deep-list-sum ( cdr p))))) 


Example 5.10: Flattening Lists 


Another way to compute thè deep list sum would be to first flatten thè List, and 
then use thè list-sum procedure. 

Flattening a nested list takes a List of Lists and evaluates to a List containing thè 
elements of thè inner Lists. We can define list-flatten by using list-append to 
append all thè inner Lists together. 
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(define ( list-flatten p) 

(if ( nuli ? p) nuli 

( list-append ( car p) ( list-flatten (cdr /;))))) 

This flattens a List of Lists into a single List. To completely flatten a deeply 
nested List, we use multiple recursive calls as we did with deep-list-sum : 

(define (deep -list-flatten p) 

(if ( nuli ? p ) nuli 

( list-append (if [lisi? ( car p)) 

[deep -list-flatten ( car /?) ) 

(list ( car p))) 

( deep -list-flatten ( cdr p ) ) ) ) ) 

Now we can define deep-list-sum as: 

(define deep-list-sum (f compose deep-list-flatten list-sum )) 


Exercise 5.24. [*] Define a procedure deep-list-map that behaves similarly to 
list-map but on deeply nested lists. It should take two parameters, a mapping 
procedure, and a List (that may contain deeply nested Lists as elements), and 
output a List with thè same structure as thè input List with each value mapped 
using thè mapping procedure. 

Exercise 5.25. [*] Define a procedure deep-list-fllter that behaves similarly to 
list-fllter but on deeply nested lists. 


Exploration 5.1: Pascal’s Triangle 


Pascal’s Triangle (named for Blaise Pascal, although known to many others be- 
fore him) is shown below: 


1 

1 1 
1 2 1 
13 3 1 
1 4 6 4 1 
1 5 10 10 5 1 



Each number in thè triangle is thè sum of thè two numbers immediately above 
and to thè left and right of it. The numbers in Pascal’s Triangle are thè coeffi- 
cients in a binomial expansion. The numbers of thè n th row (where thè rows 
are numbered starting from 0) are thè coefficients of thè binomial expansion of 
(x + y) n . For example, (x + y ) 2 — x 2 + 2 xy + y 2 , so thè coefficients are 12 1, 
matching thè third row in thè triangle; from thè fifth row, (x + 1 /) 4 — x 4 + 4 x 3 y + 
6 x 2 y 2 + 4 xy 3 + y 4 . The values in thè triangle also match thè number of ways to 
choose k elements from a set of size n (see Exercise 4.5) — thè k th number on thè 
n th row of thè triangle gives thè number of ways to choose k elements from a set 
of size n. For example, thè third number on thè fifth (n = 4) row is 6, so there are 
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6 ways to choose 3 items from a set of size 4. 

The goal of this exploration is to define a procedure, pascals-triangle to produce 
Pascal’s Triangle. The input to your procedure should be thè number of rows; thè 
output should be a list, where each element of thè list is a list of thè numbers on 
that row of Pascal’s Triangle. For example, ( pascals-triangle 0) should produce 
((1 )) (a list containing one element which is a list containing thè number 1 ), and 
[pascals-triangle 4) should produce ((1) (1 1) (1 2 1) (1 3 3 1) (1 4 6 4 1)). 

Ambitious readers should attempt to define pascals-triangle themselves; thè 
sub-parts below provide some hints for one way to define it. 

a. First, define a procedure expand-row that expands one row in thè triangle. It 
takes a List of numbers as input, and as output produces a List with one more 
element than thè input list. The first number in thè output List should be thè 
first number in thè input List; thè last number in thè output List should be 
thè last number in thè input List. Every other number in thè output List is thè 
sum of two numbers in thè input List. The n th number in thè output List is thè 
sum of thè n — l th and n th numbers in thè input List. For example, ( expand- 
row [list 1)) evaluates to (1 1); [expand-row [list 1 1)) evaluates to (12 1); and 
[expand-row [list 1 4 6 4 1)) evaluates to (1 5 10 10 5 1). This is trickier than 
thè recursive list procedures we have seen so far since thè base case is not 
thè empty list. It also needs to deal with thè first element specially. To define 
expand-row, it will be helpful to divide it into two procedures, one that deals 
with thè first element of thè list, and one that produces thè rest of thè list: 

(define [expand-row p) [cons [car p) [expand-row-rest p ) ) ) 

b. Define a procedure pascals-triangle -row that takes one input, n, and outputs 
thè n th row of Pascal’s Triangle. For example, [pascals-triangle-row 0) evalu- 
ates to (1) and ( pascals-triangle-row 3) produces (13 3 1). 

c. Finally, define pascals-triangle with thè behavior described above. 

5.6 Data Abstraction 

The mechanisms we have for constructing and manipulating complex data struc- 
tures are valuable because they enable us to think about programs closer to thè 
level of thè problem we are solving than thè low level of how data is stored and 
manipulated in thè computer. Our goal is to hide unnecessary details about how 
data is represented so we can focus on thè important aspects of what thè data 
means and what we need to do with it to solve our problem. The technique of 
data abstraction hiding how data is represented from how it is used is known as data abstraction. 

The datatypes we have seen so far are not very abstract. We have datatypes for 
representing Pairs, triples, and Lists, but we want datatypes for representing ob- 
jects closer to thè level of thè problem we want to solve. A good data abstraction 
is abstract enough to be used without worrying about details like which celi of 
thè Pair contains which datum and howto access thè different elements of a List. 
Instead, we want to define procedures with meaningful names that manipulate 
thè relevant parts of our data. 

The rest of this section is an extended example that illustrates how to solve prob- 
lems by first identifying thè objects we need to model thè problem, and then im- 
plementing data abstractions that represent those objects. Once thè appropri- 
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ate data abstractions are designed and implemented, thè solution to thè prob- 
lem often follows readily. This example also uses many of thè list procedures 
defined earlier in this chapter. 


Exploration 5.2: Pegboard Puzzle 


For this exploration, we develop a program to solve thè infamous pegboard puz- 
zle, often found tormenting unsuspecting diners at pancake restaurants. The 
standard puzzle is a one-player game played on a triangular board with fifteen 
holes with pegs in all of thè holes except one. 

The goal is to remove all but one of thè pegs by jumping pegs over one another. 
A peg may jump over an adjacent peg only when there is a free hole on thè other 
side of thè peg. The jumped peg is removed. The game ends when there are no 
possible moves. If there is only one peg remaining, thè player wins (according 
to thè Cracker Barrei version of thè game, “Leave only one — you’re genius”). If 
more than one peg remains, thè player loses (“Leave four or more’n you’re just 
plain ‘eg-no-ra-moose’.”]. 



The blue peg can jump thè red peg as shown, removing thè red peg. The resulting posi- 
tion is a winning position. 


Our goal is to develop a program that fìnds a winning solution to thè pegboard 
game from any winnable starting position. We use a brute force approach: try brute force 
all possible moves until we fmd one that works. Brute force Solutions only worlc 
on small-size problems. Because they have to try all possibilities they are of- 
ten too slow for solving large problems, even on thè most powerful computers 
imaginable . 8 

The fìrst thing to think about to solve a complex problem is what datatypes we 
need. We want datatypes that represent thè things we need to model in our 
problem solution. For thè pegboard game, we need to model thè board with its 
pegs. We also need to model actions in thè game like a move (jumping over a 
peg) . The important thing about a datatype is what you can do with it. To design 
our board datatype we need to think about what we want to do with a board. In 
thè physical pegboard game, thè board holds thè pegs. The important property 
we need to observe about thè board is which holes on thè board contain pegs. 

For this, we need a way of identifying board positions. We define a datatype 

s The generalized pegboard puzzle is an example of a class of problems known as NP-Complete. 

This means it is not known whether or not any solution exists that is substantially better than thè 
brute force solution, but it would be extraordinarily surprising (and of momentous significance!) to 
find one. 
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for representing positions first, then a datatype for representing moves, and a 
datatype for representing thè board. Finally, we use these datatypes to define a 
procedure that finds a winning solution. 

Position. We identify thè board positions using row and column numbers: 

(1 D 

(2 1 ) (2 2 ) 

(3 1) (3 2) (3 3) 

(4 1) (4 2) (4 3) (4 4) 

(5 1) (5 2) (5 3) (5 4) (5 5) 

A position has a row and a column, so we could just use a Pair to represent a 
position. This would work, but we prefer to have a more abstract datatype so we 
can think about a position’s row and column, rather than thinking that a position 
is a Pair and using thè car and cdr procedures to extract thè row and column 
from thè position. 

Our Position datatype should provide at least these operations: 

make-position: Number x Number — > Position 

Creates a Position representing thè row and column given by thè input 
numbers. 

position-get-row : Position — > Number 

Outputs thè row number of thè input Position. 
position-get-column : Position -4 Number 

Outputs thè column number of thè input Position. 

Since thè Position needs to keep track of two numbers, a naturai way to imple- 
ment thè Position datatype is to use a Pair. A more defensive implementation 
tagged list of thè Position datatype uses a tagged list. With a tagged list, thè first element 
of thè list is a tag denoting thè datatype it represents. All operations check thè 
tag is correct before proceeding. We can use any type to encode thè list tag, but 
it is most convenient to use thè built-in Symbol type. Symbols are a quote (’) 
followed by a sequence of characters. The important operation we can do with 
a Symbol, is test whether it is an exact match for another Symbol using thè eq? 
procedure. 

We define thè tagged list datatype, tlist, using thè list- get- element procedure 
from Example 5.3: 

(define ( make-tlist tag p) ( cons tag p)) 

(define [tlist- get-tag p) ( car p)) 

(define [tlist- get- element tag p n) 

(if [eq? [tlist-get-tag p) tag) 

[list- get- element [cdr p) n) 

[error [format "Bad tag: "a (expected "a)' 1 
[tlist-get-tag p) tag)))) 

The format procedure is a built-in procedure similar to thè printf procedure 
described in Section 4.5.1. Instead of printing as a side effect, format produces 
a String. For example, [format "list: "a number: “a." [list 1 2 3) 123) evaluates to thè 
String "list: (1 2 3) number: 123.". 
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This is an example of defensive programming. Using our tagged lists, if we ac- 
cidentali}? attempt to use a value that is not a Position as a position, we will get 
a clear errar message instead of a hard-to-debug errar (or worse, an unnoticed 
incorrect resulti . 

Using thè tagged list, we define thè Position datatype as: 

(define ( make-position row col) ( make-tlist 'Position ( list row col))) 

(define ( position-get-row posrì) ( tlist-get-element 'Position posn 1)) 

(define [position- get-column posn) ( tlist-get-element 'Position posn 2)) 

Here are some example interactions with our Position datatype: 

> (define pos [make-position 2 1)) 

> pos 
(Position 2 1) 

> [get-position-row pos) 

2 

> [get-position-row [list 1 2)) 

Q Bad tag: 1 (expected Position) Error since input is not a Position. 

Move. A move involves three positions: where thè jumping peg starts, thè po- 
sition of thè peg that is jumped and removed, and thè landing position. One 
possibility would be to represent a move as a list of thè three positions. A better 
option is to observe that once any two of thè positions are known, thè third po- 
sition is determined. For example, if we know thè starting position and thè land- 
ing position, we know thè jumped peg is at thè position between them. Hence, 
we could represent a jump using just thè starting and landing positions. 

Another possibility is to represent a jump by storing thè starting Position and thè 
direction. This is also enough to determine thè jumped and landing positions. 
This approach avoids thè difficulty of calculating jumped positions. To do it, we 
first design a Direction datatype for representing thè possible move directions. 
Directions have two components: thè change in thè column (we use 1 for right 
and -1 for left), and thè change in thè row (1 for down and -1 for up). 

We implement thè Direction datatype using a tagged list similarly to how we 
defined Position: 

(define [make- direction right down) 

[make-tlist 'Direction [list right down))) 

(define [direction-get-horizontal dir) [tlist-get-element 'Direction dir 1)) 
(define [direction-get-vertical dir) [tlist-get-element 'Direction dir 2)) 

The Move datatype is defined using thè starting position and thè jump direction: 

(define [make-move start direction) 

[make-tlist 'Move [list start direction))) 

(define [move-get-start move) [tlist-get-element 'Move move 1)) 

(define [move-get- direction move) [tlist-get-element 'Move move 2)) 

We also define procedures for getting thè jumped and landing positions of a 
move. The jumped position is thè result of moving one step in thè move di- 
rection from thè starting position. So, it will be useful to define a procedure that 
takes a Position and a Direction as input, and outputs a Position that is one step 
in thè input Direction from thè input Position. 
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(define ( direction-step pos dir) 

( make-position 

(+ ( posi t io n -gel- ro w pos) ( direction-get-vertical dir)) 

(+ ( position-get-column pos) [direction- get-horizontal dir)))) 

Using direction- steppe can implement procedure to get thè middle and landing 
positions. 

(define ( move- gel- jumped move) 

(i direction-step ( move-get-start move) ( move-get-direction move))) 

(define ( m ove -gel- la nd ing move) 

[direction-step [move-get-jumped move) [move-get-direction move))) 

Board. The board datatype represents thè current state of thè board. It keeps 
track of which holes in thè board contain pegs, and provides operations that 
model adding and removing pegs from thè board: 

make-board : Number — > Board 

Outputs a board full of pegs with thè input number of rows. (The stan- 
dard physical board has 5 rows, but our datatype supports any number 
of rows.) 

board-rows : Board — > Number 

Outputs thè number of rows in thè input board. 
board-valid-position ?: Board x Position — > Boolean 

Outputs true if input Position corresponds to a position on thè Board; 
otherwise, false. 

board-is-winning ?: Board -4 Boolean 

Outputs true if thè Board represents a winning position (exactly one 
peg); otherwise, false, 
board-contains-peg ?: Position Boolean 

Outputs true if thè hole at thè input Position contains a peg; otherwise, 
false. 

board-add-peg : Board x Position -4 Board 

Output a Board containing all thè pegs of thè input Board and one addi- 
tional peg at thè input Position. If thè input Board already has a peg at 
thè input Position, produces an errar. 
board-remove-peg : Board x Position — > Board 

Outputs a Board containing all thè pegs of thè input Board except for 
thè peg at thè input Position. If thè input Board does not have a peg at 
thè input Position, produces an errar. 

The procedures for adding and removing pegs change thè state of thè board to 
reflect moves in thè game, but nothing we have seen so far, however, provides a 
means for changing thè state of an existing object. 9 So, instead of defming these 
operations to change thè state of thè board, they actually create a new board that 
is different from thè input board by thè one new peg. These procedures take a 
Board and Position as inputs, and produce as output a Board. 

There are lots of different ways we could represent thè Board. One possibility is 
to keep a List of thè Positions of thè pegs on thè board. Another possibility is to 

9 We will introduce mechanisms for changing state in Chapter 9. Allowing state to change breaks 
thè substitution model of evaluation. 
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keep a List of thè Positions of thè empty holes on thè board. Yet another pos- 
sibility is to keep a List of Lists, where each List corresponds to one row on thè 
board. The elements in each of thè Lists are Booleans representing whether or 
not there is a peg at that position. The good thing about data abstraction is we 
could pick any of these representations and change it to a different representa- 
tion later (for example, if we needed a more efficient board implementation). As 
long as thè procedures for implementing thè Board are updated thè work with 
thè new representation, all thè code that uses thè board abstraction should con- 
tinue to work correctly without any changes. 

We choose thè third option and represent a Board using a List of Lists where 
each element of thè inner lists is a Boolean indicating whether or not thè cor- 
responding position contains a peg. So, make-board evaluates to a List of Lists, 
where each element of thè List contains thè row number of elements and all thè 
inner elements are true (thè initial board is completely full of pegs). First, we de- 
fine a procedure make-list-of-constants that takes two inputs, a Number, n, and 
a Value, vai. The output is a List of length n where each element has thè value 
vai. 

(define ( make-list-of-constants n vai ) 

(if (= nO) nuli ( cons vai ( make-list-of-constants (—ni) vai)))) 

To make thè initial board, we use make-list-of-constants to make each row of thè 
board. As usuai, a recursive problem solving strategy works well: thè simplest 
board is a board with zero rows (represented as thè empty list); for each larger 
board, we add a row with thè right number of elements. 

The tricky part is putting thè rows in order. This is similar to thè problem we 
faced with intsto, and a similar solution using append-list works here: 

(define ( make-board rows) 

(if (= rows 0) nuli 

( list-append ( make-board (— rows 1)) 

[list ( make-list-of-constants rows true))))) 

Evaluating [make-board 3) produces [[true) [true true) [true true true)). 

The board-rows procedure takes a Board as input and outputs thè number of 
rows on thè board. 

(define [board-rows board) [length board)) 

The board-valid-position? indicates if a Position is on thè board. A position is 
valid if its row number is between 1 and thè number of rows on thè board, and 
its column numbers is between 1 and thè row number. 

(define [board-valid-position? board pos) 

[and (>= [position-get-row pos) 1) (>= [position-get-column pos) 1) 

(<= [position-get-row pos) [board-rows board)) 

[<— [position-get-column pos) [position-get-row pos)))) 

Weneedawayto check if a Board represents a winning solution (thatis, contains 
only one peg) . We implement a more generai procedure to count thè number of 
pegs on a board first. Our board representation used true to represent a peg. 
To count thè pegs, we first map thè Boolean values used to represent pegs to 
1 if there is a peg and 0 if there is no peg. Then, we use sum-list to count thè 
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number of pegs. Since thè Board is a List of Lists, we fìrst use list-flatten to put 
all thè pegs in a single List. 

(define [board- ri u mber- o f- pegs board) 

( list-sum 

(, list-map (lambda [peg) (if peg 1 0)) ( list-flatten board)))) 

A board is a winning board if it contains exactly one peg: 

(define ( board-is-wìnning ? board) 

(= ( board-number-of-pegs board) 1)) 

The board-contains-peg? procedure takes a Board and a Position as input, and 
outputs a Boolean indicating whether or not that Position contains a peg. To im- 
plement board-contains-peg? we need to fìnd thè appropriate row in our board 
representation, and then find thè element in its list corresponding to thè posi- 
tion’s column. The list- get- element procedure (from Example 5.3) does exactly 
what we need. Since our board is represented as a List of Lists, we need to use it 
twice: first to get thè row, and then to select thè column within that row: 

(define ( board-contains-peg ? board pos) 

[list- get- element ( list- get- element board [position-get-row pos)) 
[position-get-column pos))) 

Defining procedures for adding and removing pegs from thè board is more com- 
plicated. Both of these procedures need to make a board with every row identi- 
cal to thè input board, except thè row where thè peg is added or removed. For 
that row, we need to replace thè corresponding value. Hence, instead of defining 
separate procedures for adding and removing we first implement a more generai 
board-replace-peg procedure that takes an extra parameter indicating whether 
a peg should be added or removed at thè selected position. 

First we consider thè subproblem of replacing a peg in a row. The procedure 
row-replace-peg takes as input a List representing a row on thè board and a 
Number indicating thè column where thè peg should be replaced. We can de- 
fìne row-replace-peg recursively: thè base case is when thè modifìed peg is at thè 
beginning of thè row (thè column number is 1); in thè recursive case, we copy 
thè first element in thè List, and replace thè peg in thè rest of thè list. The third 
parameter indicates if we are adding or removing a peg. Since true values rep- 
resent holes with pegs, a true value indicates that we are adding a peg and false 
means we are removing a peg. 

(define [row-replace-peg pegs col vai) 

(if (— col 1 ) 

[cons vai [cdr pegs)) 

[cons [car pegs) [row-replace-peg [cdr pegs) (- col 1) vai)))) 

To replace thè peg on thè board, we use row-replace-peg to replace thè peg on 
thè appropriate row, and keep all thè other rows thè same. 

(define [board-replace-peg board row col vai) 

(if (— row 1 ) 

[cons [row-replace-peg [car board) col vai) [cdr board)) 

[cons [car board) [board-replace-peg [cdr board ) [— row 1) col vai)))) 

Both board-add-peg and board-remove-peg can be defined simply using board- 
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remove-peg. They first check if thè operation is valid (adding is valid only if thè 
selected position does not contain a peg, removing is valid only if thè selected 
position contains a peg), and then use board-replace-peg to produce thè modi- 
fied board: 

(define ( board-add-peg board pos ) 

(if ( board-contains-peg ? board pos) 

( error {format "Board already contains peg at position: “a" pos)) 
{board-replace-peg board {position-get-row pos) 

{position-get-column pos) true ))) 

(define {board-remove-peg board pos) 

(if {not {board-contains-peg? board pos)) 

{error {format "Board does not contain peg at position: ~a" pos)) 
{board-replace-peg board {position-get-row pos) 

{position-get-column pos) false))) 

We can now define a procedure that models making a move on a board. Making 
a move involves removing thè jumped peg and moving thè peg from thè start- 
ing position to thè landing position. Moving thè peg is equivalent to removing 
thè peg from thè starting position and adding a peg to thè landing position, so 
thè procedures we defined for adding and removing pegs can be composed to 
model making a move. We add a peg landing position to thè board that results 
from removing thè pegs in thè starting and jumped positions: 

(define {board-execute-move board move) 

( board-add-peg 
{board-remove-peg 

{board-remove-peg board {move-get-start move)) 

{move-get-jumped move)) 

{move-get-landing move))) 

Finding Valid Moves. Now that we can model thè board and simulate making 
jumps, we are ready to develop thè solution. At each step, we try all valid moves 
on thè board to see if any move leads to a winning position (that is, a position 
with only one peg remaining) . So, we need a procedure that takes a Board as 
its input and outputs a List of all valid moves on thè board. We break this down 
into thè problem of producing a list of all conceivable moves (all moves in all 
directions from all starting positions on thè board), filtering that list for moves 
that stay on thè board, and then filtering thè resulting list for moves that are legai 
(start at a position containing a peg, jump over a position containing a peg, and 
land in a position that is an empty hole). 

First, we generate all conceivable moves by creating moves starting from each 
position on thè board and moving in all possible move directions. We break this 
down further: thè first problem is to produce a List of all positions on thè board. 

We can generate a list of all row numbers using thè intsto procedure (from Ex- 
ampie 5.8). To get a list of all thè positions, we need to produce a list of thè 
positions for each row. We do this by mapping each row to thè corresponding 
list: 
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(define ( all- posi t io ns - he l per board) 

( list-map 

(lambda ( row ) ( list-map (lambda (col) (make-position row col)) 

(intsto row))) 

( intsto ( board-rows board))) 

This almost does what we need, except instead of producing one List containing 
all thè positions, it produces a List of Lists for thè positions in each row. The 
list-flatten procedure (from Example 5.10) produces a fiat list containing all thè 
positions. 

(define ( all-positions board) 

(list-flatten (all-positions-helper board))) 

For each Position, we find all possible moves starting from that position. We 
can move in six possible directions on thè board: left, right, up-left, up-right, 
down-left, and down-right. 

(define all- directions 
(list 

(make-direction -1 0) (make- direction 1 0) ; left, right 
(make-direction -1-1) (make-direction 0 -1) ; up-left, up-right 
(make-direction 0 1) (make-direction 1 1))) ; down-left, down-right 

For each position on thè board, we create a list of possible moves starting at that 
position and moving in each possible move directions. This produces a List of 
Lists, so we use list-flatten to flatten thè output of thè list-map application into 
a single List of Moves. 

(define (all-conceivable-moves board) 

(list-flatten 

(list-map 

(lambda ( pos ) (list-map (lambda (dir) (make-move pos dir)) 

all-directions)) 

(all-positions board)))) 

The output produced by all-conceivable-moves includes moves that fly off thè 
board. We use thè list-fìlter procedure to remove those moves, to get thè list of 
moves that stay on thè board: 

(define (all-board-moves board) 

(list-fìlter 

(lambda (move) (board-valid-position? board (move-get-landing move))) 
(all-conceivable-moves board))) 

Finally, we need to filter out thè moves that are not legai moves. A legai move 
must start at a position that contains a peg, jump over a position that contains a 
peg, and land in an empty hole. We use list-fìlter similarly to how we kept only 
thè moves that stay on thè board: 


Chapter5. Data 


101 


(define (all- legai- moves board ) 

(lisi- filler 
(lambda (move) 

(and 

(board-contains-peg? board (move-get-start move)) 
(board-contains-peg? board (move-get-jumped move)) 

(not (board-contains-peg? board (move-get-landing move))))) 
(all-board-moves board))) 

Winning thè Game. Our goal is to find a sequence of moves that leads to a win- 
ning position, starting from thè current board. If there is a winning sequence of 
moves, we can find it by trying all possible moves on thè current board. Each of 
these moves leads to a new board. If thè originai board has a winning sequence 
of moves, at least one of thè new boards has a winning sequence of moves. 
Hence, we can solve thè puzzle by recursively trying all moves until fìnding a 
winning position. 

(define (solve-pegboard board) 

(if (board-is-winning? board) 

nuli ; no moves needed to reach winning position 
( try-moves board (all-legal-moves board)))) 

If there is a sequence of moves that wins thè game starting from thè input Board, 
solve-pegboard outputs a List of Moves representing a winning sequence. This 
could be nuli, in thè case where thè input board is already a winning board. If 
there is no sequence of moves to win from thè input board, solve-pegboard out- 
puts false. 

It remains to defìne thè try-moves procedure. It takes a Board and a List of Moves 
as inputs. If there is a sequence of moves that starts with one of thè input moves 
and leads to a winning position it outputs a List of Moves that wins; otherwise, 
it outputs false. 

The base case is when there are no moves to try. When thè input list is nuli 
it means there are no moves to try. We output false to mean this attempt did 
not lead to a winning board. Otherwise, we try thè first move. If it leads to a 
winning position, try-moves should output thè List of Moves that starts with thè 
first move and is followed by thè rest of thè moves needed to solve thè board 
resulting from taking thè first move (that is, thè result of solve-pegboard applied 
to thè Board resulting from taking thè first move). If thè first move doesn’t lead 
to a winning board, it tries thè rest of thè moves by calling try-moves recursively. 

(define (try-moves board moves) 

(if (nuli? moves) 

false ; didn’t find a winner 

(if (solve-pegboard (board-execute-move board (car moves))) 

(cons (car moves) 

(solve-pegboard (board-execute-move board (car moves)))) 
(try-moves board (cdr moves))))) 

Evaluating (solve-pegboard (make-board 5)) produces false since there is no way 
to win starting from a completely full board. Evaluating (solve-pegboard (board- 
remove-peg (make-board 5) (make-position 1 1))) takes about three minutes to 
produce this sequence of moves for winning thè game starting from a 5-row 
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board with thè top peg removed: 

(( Move ( Position 3 1) ( Direction 0-1)) 

{Move { Position 3 3) ( Direction -10)) 

{Move {Position 1 1) {Direction 1 1)) 

{Move {Position 4 1) ( Direction 0-1)) 

... ; 8 moves elided 

{Move {Position 5 1) ( Direction 1 1))) 

a. [*] Change thè implementation to use a different Board representation, such 
as keeping a list of thè Positions of each hole on thè board. Only thè proce- 
dures with names starting with board- should need to change when thè Board 
representation is changed. Compare your implementation to this one. 

b. [*] The standard pegboard puzzle uses a triangular board, but there is no 
reason thè board has to be a triangle. Define a more generai pegboard puzzle 
solver that works for a board of any shape. 

c. [*★] The described implementation is very ineffìcient. It does lots of redun- 
dant computation. For example, all-possible-moves evaluates to thè same 
value every time it is applied to a board with thè same number of rows. It is 
wasteful to recompute this over and over again to solve a given board. See 
how much faster you can make thè pegboard solver. Can you make it fast 
enough to solve thè 5-row board in less than half thè originai time? Can you 
make it fast enough to solve a 7-row board? 


5.7 Summary of Part I 

To conclude Part I, we revisit thè three main themes introduced in Section 1.4. 

Recursive deflnitions. We have seen many types of recursive definitions and used 
them to solve problems, including thè pegboard puzzle. Recursive grammars 
provide a compact way to define a language; recursive procedure definitions 
enable us to solve problems by optimistically assuming a smaller problem in- 
stance can be solved and using that solution to solve thè problem; recursive data 
structures such as thè list type allow us to define and manipulate complex data 
built from simple components. All recursive definitions involve a base case. For 
grammars, thè base case provides a way to stop thè recursive replacements by 
produce a terminal (or empty output) directly; for procedures, thè base case 
provides a direct solution to a small problem instance; for data structures, thè 
base case provides a small instance ofthe data type (e.g., nuli). We will see many 
more examples of recursive definitions in thè rest of this hook. 

Universality. All of thè programs we have can be created from thè simple subset 
of Scheme introduced in Chapter 3. This subset is a universal programming 
language-. it is powerful enough to describe all possible computations. We can 
generate all thè programs using thè simple Scheme grammar, and interpret their 
meaning by systematically following thè evaluation rules. We have also seen thè 
universality of code and data. Procedures can take procedures as inputs, and 
produce procedures as outputs. 

Abstraction. Abstraction hides details by giving things names. Procedural ab- 
straction defines a procedure; by using inputs, a short procedure definition can 
abstract infinitely many different information processes. Data abstraction hides 
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thè details of how data is represented by providing procedures that abstractly 
create and manipulate that data. As we develop programs to solve more com- 
plex problems, it is increasingly important to use abstraction well to manage 
complexity. We need to break problems down into smaller parts that can be 
solved separately. Solutions to complex problems can be developed by think- 
ing about what objects need to be modeled, and designing data abstractions thè 
implement those models. Most of thè work in solving thè problem is debning 
thè right datatypes; once we have thè datatypes we need to model thè problem 
well, we are usually well along thè path to a solution. 

With thè tools from Part I, you can debne a procedure to do any possible com- 
putation. In Part II, we examine thè costs of executing procedures. 
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Machines 


It is unworthy ofexcellent people to lose hours lìke slaves in thè labor of 
calculation which could safely be relegateci to anyone else ifmachines mere used. 

Gottfried Wilhelm von Leibniz, 1685 


The first five chapters focused on ways to use language to describe procedures. 
Although finding ways to describe procedures succinctly and precisely would 
be worthwhile even if we did not have machines to carry out those procedures, 
thè tremendous practical value we gain from being able to describe procedures 
Comes from thè ability of computers to carry out those procedures astoundingly 
quickly, reliably, and inexpensively. As a very rough approximation, a typical 
laptop gives an individuai computing power comparable to having every living 
human on thè planet working for you without ever making a mistake or needing 
a break. 

This chapter introduces computing machines. Computers are different from 
other machines in two key ways: 

1. Whereas other machines amplify or extend our physical abilities, comput- 
ers amplify and extend our meritai abilities. 

2. Whereas other machines are designed for a few specific tasks, computers 
can be programmed to perform many tasks. The simple computer model 
introduced in this chapter can perform all possible computations. 


The next section gives a brief history of computing machines, from prehistoric 
calculating aids to thè design of thè first universal computers. Section 6.2 ex- 
plains how machines can implement logie. Section 6.3 introduces a simple ab- 
stract model of a computing machine that is powerful enough to carry out any 
algorithm. 

We provide only a very shallow introduction to how machines can implement 
computations. Our primary goal is not to convey thè details of how to design 
and build an efficient computing machine (although that is certainly a worthy 
goal that is often pursued in later computing courses), but to gain sufficient un- 
derstanding of thè properties nearly all conceivable computing machines share 
to be able to predict properties about thè costs involved in carrying out a par- 
ticular procedure. The following chapters use this to reason about thè costs of 
various procedures. In Chapter 12, we use it to reason about thè range of prob- 
lems that can and cannot be solved by any mechanical computing machine. 
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6. 1 History of Computing Machines 

The goal of early machines was to carry out some physical process with less ef- 
fort than would be required by a human. These machines took physical things as 
inputs, performed physical actions on those things, and produced some phys- 
ical output. For instance, a cotton gin takes as input raw cotton, mechanically 
separates thè cotton seed and lint, and produces thè separated products as out- 
put. 

The first big leap toward computing machines was thè development of machines 
whose purpose is abstract rather than physical. Instead of producing physical 
things, these machines used physical things to rep rese ni i n fo r m ation. The out- 
put of thè machine is valuable because it can be interpreted as information, not 
for its direct physical effect. 

Our first example is not a machine, but using fingers to count. The base ten 
number System used by most human cultures reflects using our ten fingers for 
counting. 1 Successful shepherds needed to find ways to count higher than ten. 
Shepherds used stones to represent numbers, making thè cognitive leap of using 
a physical stone to represent some quantity of sheep. A shepherd would count 
sheep by holding stones in his hand that represent thè number of sheep. 



Suan Pan 


More complex societies required more counting and more advanced calculat- 
ing. The Inca civilization in Perù used knots in collections of strings known as 
khipu to keep track of thousands of items for a hierarchical System of taxation. 
Many cultures developed forms of abaci, including thè ancient Mesopotamians 
and Romans. An abacus performs calculations by moving beads on rods. The 
Chinese suan pan (“calculating piate”) is an abacus with a beam subdividing 
thè rods, typically with two beads above thè bar (each representing 5), and five 
beads below thè beam (each representing 1). An operator can perform addition, 
subtraction, multiplication, and division by following mechanical processes us- 
ing an abacus. 



Pascaline 

David Monniaux 


All of these machines require humans to move parts to perform calculations. 
As machine technology improved, automatic calculating machines were built 
where thè operator only needed to set up thè inputs and then turn a crank or 
use some external power source to perform thè calculation. The first automatic 
calculating machine to be widely demonstrated was thè Pascaline, built by then 
nineteen-year old French mathematician Blaise Pascal (also responsible for Pas- 
cali triangle from Exploration 5.1) to replace thè tedious calculations he had to 
do to manage his father’s accounts. The Pascaline had five wheels, each repre- 
senting one digit of a number, linked by gears to perform addition with carries. 
Gottfried Wilhelm von Leibniz built thè first machine capable of performing all 
four basic arithmetic operations (addition, subtraction, multiplication, and di- 
vision) fully mechanically in 1694. 


Over thè following centuries, more sophisticated mechanical calculating ma- 
chines were developed but these machines could stili only perform one opera- 
tion at a time. Performing a series of calculations was a tedious and error-prone 
process in which a human operator had to set up thè machine for each arith- 


x Not all human cultures use base ten number Systems. For example, many cultures including thè 
Maya and Basque adopted base twenty Systems counting both fingers and toes. This was naturai in 
warm areas, where typical footwear left thè toes uncovered. 
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metic operation, record thè result, and reset thè machine for thè next calcula- 
tion. 

The big breakthrough was thè conceptual leap of programmability. A machine 
is programmable if its inputs not only control thè values it operates on, but thè 
operations it performs. 

The first programmable computing machine was envisioned (but never suc- 
cessfully built) in thè 1830s by Charles Babbage. Babbage was born in London 
in 1791 and studied mathematics at Cambridge. In thè 1800s, calculations were 
done by looking up values in large books of mathematical and astronomical ta- 
bles. These tables were computed by hand, and often contained errors. The 
calculations were especially important for astronomical navigation, and when 
thè values were incorrect a ship would miscalculate its position at sea (some- 
times with tragic consequences). 

Babbage sought to develop a machine to mechanize thè calculations to compute 
these tables. Starting in 1822, he designed a steam-powered machine known 
as thè Difference Engine to compute polynomials needed for astronomical cal- 
culations using Newton’s method of successive differences (a generalization of 
Heron’s method from Exploration 4.1). The Difference Engine was never fully 
completed. but led Babbage to envision a more generai calculating machine. 

This new machine, thè Analytical Engine, designed between 1833 and 1844, was 
thè first general-purpose computer envisioned. It was designed so that it could 
be programmed to perform any calculation. One breakthrough in Babbage’s de- 
sign was to feed thè machine’s outputs back into its inputs. This meant thè en- 
gine could perform calculations with an arbitrary number of steps by cycling 
outputs back through thè machine. 

The Analytical Engine was programmed using punch cards, based on thè cards 
that were used by Jacquard looms. Each card could describe an instruction such 
as loading a number into a variable in thè store, moving values, performing 
arithmetic operations on thè values in thè store, and, most interestingly, jump- 
ing forward and backwards in thè instruction cards. The Analytical Engine sup- 
ported conditional jumps where thè jump would be taken depending on thè 
state of a lever in thè machine (this is essentially a simple form of thè if expres- 
sion). 

In 1842, Charles Babbage visited Italy and described thè Analytical Engine to 
Luigi Menabrea, an Italian engineer, military officer, and mathematician who 
would later become Prime Minister of Italy. Menabrea published a description 
of Babbage’s lectures in French. Ada Augusta Byron King (also known as Ada, 
Countess of Lovelace) translated thè article into English. 

In addition to thè translation, Ada added a series of notes to thè article. The 
notes included a program to compute Bernoulli numbers, thè first detailed pro- 
gram for thè Analytical Engine. Ada was thè first to realize thè importance and 
interest in creating thè programs themselves, and envisioned how programs 
could be used to do much more than just calculate mathematical functions. 
This was thè first computer program ever described, and Ada is recognized as 
thè first computer programmer. 

Despite Babbage’s design, and Ada’s vision, thè Analytical Engine was never com- 


We got nothingfor 
our £17,000 butMr. 
Babbage ’s 
grumblings. We 
should at least bave 
had a clever toyfor 
our money. 

Richard Sheepshanks, 
Letter to thè Board of 
Visitors ofthe 
Greenwich Royal 
Observatory, Ì854 



Analytical Engine 

Science Museum, London 



Ada 
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pleted. It is unclear whether thè main reason for thè failure to build a working 
Analytical Engine was due to limitations of thè mechanical components avail- 
able at thè tinte, or due to Babbage’s inability to work with his engineer collabo- 
On two occasionsi rator or to secure continued funding. 


have been asked by 
members of 
Parliament, “Pray, 
Mr. Babbage, ifyou 
put into thè 
machine wrong 
figures, will thè 
right answers come 


The first working programmable computers would not appear for nearly a hun- 
dred years. Advances in electronics enabled more reliable and faster compo- 
nents than thè mechanical components used by Babbage, and thè desperation 
brought on by World War II spurred thè funding and efforts that led to working 
general-purpose computing machines. 


out?" I am notable 
rightly to 
apprehend thè kind 
ofconfusion ofideas 
that could provoke 
such a question. 
Charles Babbage 


The remaining conceptual leap is to treat thè program itself as data. In Bab- 
bage’s Analytical Engine, thè program is a stack of cards and thè data are num- 
bers stored in thè machine. The machine cannot alter its own program. 

The idea of treating thè program as just another kind of data thè machine can 
process was developed in theory by Alan Turing in thè 1930s (Section 6.3 of 
this chapter describes his model of computing), and first implemented by thè 
Manchester Small-Scale Experimental Machine (built by a team at Victoria Uni- 
versity in Manchester) in 1948. 


This computer (and all general-purpose computers in use today) Stores thè pro- 
gram itself in thè machine’s memory. Thus, thè computer can create new pro- 
grams by writing into its own memory. This power to change its own program is 
what makes stored-program computers so versatile. 


Exercise 6.1. Babbage’s design for thè Analytical Engine called for a store hold- 
ing 1000 variables, each of which is a 50-digit (decimai) number. How many bits 
could thè store of Babbage’s Analytical Engine hold? 


6.2 Mechanizing Logic 

This section explains how machines can compute, starting with simple logicai 
Boolean logie operations. We use Boolean logie, in which there are two possible values: true 
(often denoted as 1), and false (often denoted as 0). The Boolean datatype in 
Scheme is based on Boolean logie. Boolean logie is named for George Boole, a 
self-taught British mathematician who published An investigation into theLaws 
ofThought, on Which are founded thè Mathematical Theories of Logic and Prob- 
abilities in 1854. Before Boole’s work, logie focused on naturai language dis- 
course. Boole made logie a formai language to which thè tools of mathematics 
could be applied. 


George Boole 


We illustrate how logicai functions can be implemented mechanically by de- 
scribing some logicai machines. Modern computers use electrons to compute 
because they are small (more than a billion billion billion (IO 31 ) electrons fit 
within thè volume of a grain of sand), fast (approaching thè speed of light), and 
cheap (more than a billion billion (IO 22 ) electrons come out of a power outlet for 
less than a cent). They are also invisible and behave in somewhat mysterious 
ways, however, so we will instead consider how to compute with wine (or your 
favorite colored liquid). The basic notions of mechanical computation don’t de- 
pend on thè medium we use to compute, only on our ability to use it to represent 
values and to perform simple logicai operations. 
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6.2.1 Implementing Logic 

To implement logie using a machine, we need physical ways of representing thè 
two possible values. We use a full bottle of wine to represent true and an empty 
bottle of wine to represent false. If thè value of an input is Ime, we pour a bottle 
of wine in thè input nozzle; for false inputs we do nothing. Similarly, electronic 
computers typically use presence of voltage to represent ime, and absence of 
voltage to represent false. 

And. A logicai and function takes two inputs and produces one output. The 
output is true if both of thè inputs are trae ; otherwise thè output is false. We 
defìne a logical-and procedure using an if expression: 2 

(define ( logical-and a b) (if a b false)) 

To design a mechanical implementation of thè logicai and function, we want a 
simpler definition that does not involve implementing something as complex as 
an if expression. 

A different way to define a function is by using a table to show thè corresponding 
output value for each possible pair of input values. This approach is limited to 
functions with a small number of possible inputs; we could not define addition 
on integers this way, since there are infinitely many possible different numbers 
that could be used as inputs. For functions in Boolean logie, there are only two 
possible values for each input ( true and false) so it is feasible to list thè outputs 
for all possible inputs. 

We cali a table defining a Boolean function a trulli table. If there is one input, thè inali table 
table needs two entries, showing thè output value for each possible input. When 
there are two inputs, thè table needs four entries, showing thè output value for 
all possible combinations of thè input values. The truth table for thè logicai and 
function is: 


A 

B 

(i and A B) 

false 

false 

false 

true 

false 

false 

false 

true 

false 

true 

true 

true 


We design a machine that implements thè function described by thè truth ta- 
ble: if both inputs are true (represented by full bottles of wine in our machine), 
thè output should be true ; if either input is false, thè output should be false (an 
empty bottle). One way to do this is shown in Figure 6.1. Both inputs pour into 
a basin. The output nozzle is placed at a height corresponding to one bottle of 
wine in thè collection basin, so thè output bottle will fili (representing true), only 
if both inputs are true. 

The design in Figure 6. 1 would probably not work very well in practice. Some 
of thè wine is likely to spili, so even when both inputs are true thè output might 
not be a full bottle of wine. What should a \ full bottle of wine represent? What 
about a bottle that is half full? 

2 Scheme provides a special form and that performs thè sartie function as thè logicai and function. 
It is a special form, though, since thè second input expression is not evaluated unless thè fìrst input 
expression evaluates to true. 
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Figure 6.1. Computing and with wine. 

digitai abstraction The solution is thè digitai abstraction. Although there are many different quan- 
tities of wine that could be in a bottle, regardless of thè actual quantity thè value 
is interpreted as only one of two possible values: true or false. If thè bottle has 
more than a given threshold, say half full, it represents true ; otherwise, it repre- 
sents false. This means an infrnitely large set of possible values are abstracted as 
meaning true, so it doesn’t matter which of thè values above half full it is. 

The digitai abstraction provides a transition between thè continuous world of 
physical things and thè logicai world of discrete values. It is much easier to de- 
sign computing Systems around discrete values than around continuous values; 
by mapping a range of possible continuous values to just two discrete values, we 
give up a lot of information but gain in simplicity and reliability. Nearly all com- 
puting machines today operate on discrete values using thè digitai abstraction. 

Or. The logicai or function takes two inputs, and outputs true if any of thè 
inputs are true: 3 


A 

B 

{or AB) 

false 

false 

false 

true 

false 

true 

false 

true 

true 

true 

true 

true 


Try to invent your own design for a machine that computes thè or function be- 
fore looking at one solution in Figure 6.2 (a). 

Implementing not. The output of thè noi function is thè opposite of thè value 
of its input: 


A 

{not A) 

false 

true 

true 

false 


3 Scheme provides a special form or that implements thè logicai or function, similarly to thè and 
special form. If thè first input evaluates to true, thè second input is not evaluated and thè value of 
thè or expression is true. 
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It is not possible to produce a logicai not without some other source of wine; it 
needs to create wine (to represent true) when there is none input (representing 
false). To implement thè not function, we need thè notion of a source current 
and a clock. The source current injects a bottle of wine on each clock tick. The 
clock ticks periodically, on each operation. The inputs need to be set up before 
thè clock tick. When thè clock ticks, a bottle of wine is sent through thè source 
current, and thè output is produced. Figure 6.2 (b) shows one way to implement 
thè not function. 


N /♦ 




(a) Computing or with wine. (b) Computing not with wine. 

Figure 6.2. Computing logicai or and not with wine 

(a) The or machine is similar to thè and machine in design, except we move thè output nozzle 
to thè bottom of thè basin, so if either input is true, thè output is true-, when both inputs are 
true, some wine is spilled but thè logicai result is stili trite. 

(b) The not machine uses a clock. Before thè clock tick, thè input is set. If thè input is true, thè 
float is lifted, blocking thè source opening; if thè input i false, thè float rests on thè bottom of 
thè basin. When thè clock ticks, thè source wine is injected. If thè float is up (because of thè true 
input), thè opening is blocked, and thè output is empty (false). If thè float is down (because of 
thè false input) , thè opening is open, thè source wine will pour across thè float, filling thè output 
(representing true) . (This design assumes wine coming from thè source does not leak under thè 
float, which might be hard to build in a reai System.) 


6.2.2 Composing Operations 

We can implement and, or and not using wine, but is that enough to perform 
interesting computations? In this subsection, we consider how simple logicai 
functions can be combined to implement any logicai function; in thè following 
subsection, we see how basic arithmetic operations can be built from logicai 
functions. 

We start by making a three-input conjunction function. The and3 of three inputs 
is true if and only if all three inputs are true. One way to make thè three-input 
and3 is to follow thè same idea as thè two-input and where all three inputs pour 
into thè same basin, but make thè basin with thè output nozzle above thè two 
bottle level. 

Another way to implement a three-input and3 is to compose two of thè two- 
input and functions, similarly to how we composed procedures in Section 4.2. 
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Building and3 by composing two two-input and functions allows us to construct 
a three-input and3 without needing to design any new structures, as shown in 
Figure 6.3. The output of thè first and function is fed into thè second and func- 
tion as its first input; thè third input is fed directly into thè second and function 
as its second input. We could write this as (and (and AB) C ). 


\ / / 



Figure 6.3. Computing and3 by composing two and functions. 

Composing logicai functions also allows us to build new logicai functions. Con- 
sider thè xor (exclusive or) function that takes two inputs, and has output true 
when exactly one of thè inputs is true: 


A 

B 

(xor A B ) 

false 

false 

false 

true 

false 

true 

false 

true 

true 

true 

true 

false 


Can we build xor by composing thè functions we already have? 

The xor is similar to or, except for thè result when both inputs are true. So, we 
could compute (xor A B ) as (and (or A B) (not (and A B))). Thus, we can build 
an xor machine by composing thè designs we already have for and, or, and not. 

We can compose any pair of functions where thè outputs for thè first function 
are consistent with thè input for thè second function. One particularly impor- 
tant function known as nand results from not and and: 


A 

B 

(nand A B) 

false 

false 

true 

true 

false 

true 

false 

true 

true 

true 

true 

false 
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All Boolean logie functions can be implemented using just thè nand function. 
One way to prove this is to show how to build all logie functions using just nand. 
For example, we can implement not using nand where thè one input to thè noi 
function is used for both inputs to thè nand function: 

[not A) = [nand A A) 

Now that we have shown how to implement not using nand, it is easy to see how 
to implement and using nand: 

land AB) = ( not ( nand A B ')) 

Implementing or is a bit trickier. Recali tirati or B is true if any one of thè inputs 
is true. But, A nand B is true if both inputs are false, and false if both inputs are 
true. To compute or using only nand functions, we need to invert both inputs: 

[or AB) = ( nand [not A) [not B)) 

To complete thè proof, we would need to show how to implement all thè other 
Boolean logie functions. We omit thè details here, but leave some of thè other 
functions as exercises. The universality of thè nand function makes it very useful 
for implementing computing devices. Trillions of nand gates are produced in 
Silicon every day. 

Exercise 6.2. Define a Scheme procedure, logical-or, that takes two inputs and 
outputs thè logicai or of those inputs. 

Exercise 6.3. What is thè meaning of composing not with itself? For example, 
( not [not A)). 

Exercise 6.4. Define thè xor function using only nand functions. 

Exercise 6.5. [*] Our defmition of ( not A) as ( nand A A) assumes there is a 

way to produce two copies of a given input. Design a component for our wine 
machine that can do this. It should take one input, and produce two outputs, 
both with thè same value as thè input. (Hint: when thè input is true, we need 
to produce two full bottles as outputs, so there must be a source similarly to thè 
not component.) 

Exercise 6.6. [*] The digitai abstraction works fine as long as actual values stay 
dose to thè value they represent. But, if we continue to compute with thè out- 
puts of functions, thè actual values will get increasingly fuzzy. For example, if 
thè inputs to thè and3 function in Figure 6.3 are initially all | full bottles (which 
should be interpreted as true), thè basin for thè first and function will fili Lo 1 j , 
so only ^ bottle will be output from thè first and. When combined with thè third 
input, thè second basin will contain 1 \ bottles, so only \ will spili into thè output 
bottle. Thus, thè output will represent false, even though all three inputs repre- 
sent true. The solution to this problem is to use an amplifler to restore values to 
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their full representations. Design a wine machine amplifìer that takes one input 
and produces a strong representation of that input as its output. If that input 
represents true (any value that is half full or more), thè amplifìer should output 
true, but with a strong, full bottle representation. If that input represents false 
(any value that is less than half full), thè amplifìer should output a strong false 
value (completely empty) . 

6.2.3 Arithmetic 

Not only is thè nand function complete for Boolean logicai functions, it is also 
enough to implement all discrete arithmetic functions. First, consider thè prob- 
lem of adding two one-bit numbers. 

There are four possible pairs of inputs: 

A B ri ro 

“0 + Ò = 0 ò - 

0 + 1 = 0 1 

1+0 = 0 1 
1 + 1 = 1 0 

We can compute each of thè two output bits as a logicai function of thè two input 
bits. The right output bit, tq, is 1 if exactly one of thè input bits is 1 : 

r 0 = ( or ( and [not A) B) [and A {not B))) 

This is what thè xor function computes, so: 

ro = {xor A B) 

The left output bit, r 1; is 0 for all inputs except when both inputs are 1 : 

r\ = {and A B) 

Since we have already seen how to implement and, or, xor, and not using only 
nand functions, this means we can implement a one-bit adder using only nand 
functions. 

Adding larger numbers requires more logicai functions. Consider adding two 
n-bit numbers: 


«n— 1 a n-2 ' ' ' Cl\ «0 

+ fc„_i b n - 2 ■■■ h b 0 


= r n r n _\ r n _ 2 ■■■ D r 0 


The elementary school algorithm for adding decimai numbers is to sum up thè 
digits from right to left. If thè result in one place is more than one digit, thè 
additional tens are carried to thè next digit. We use cy to represent thè carry digit 
in thè k th column. 
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Cji— 1 Cn—2 


Q-n — 1 ®n— 2 

bn—1 b n —2 


= r n r n _\ r n _2 ■■■ n r 0 


The algorithm for addition is: 

• Initially, c 0 = 0. 

• Repeat for each digit k from 0 to n: 

1. v\Vq — a k + b k + c k (if there is no digit a k or b k use 0). 

2. r k = v 0 . 

3 - c k +i Vi- 

This is perhaps thè first interesting algorithm most people learn: if followed cor- 
rectly, it is guaranteed to produce thè correct result, and to always finish, for any 
two input numbers. 

Step 1 seems to require already knowing how to perform addition, since it uses 
+. But, thè numbers added are one-digit numbers (and c k is 0 or 1). Hence, there 
are a finite number of possible inputs for thè addition in step 1:10 decimai digits 
for a k x 10 decimai digits for b k x 2 possible values of c k . We can memorize thè 
100 possibilities for adding two digits (or write them down in a table), and easily 
add one as necessary for thè carry. Hence, computing this addition does not 
require a generai addition algorithm, just a specialized method for adding one- 
digit numbers. 

We can use thè same algorithm to sum binary numbers, except it is simpler since 
there are only two binary digits. Without thè carry bit, thè result bit, r k , is 1 if ( xor 
a k b k ) . If thè carry bit is 1 , thè result bit should flip. So, 

r k = ( xor ( xor a k b k ) c k ) 

This is thè same as adding a k + b k + c k base two and keeping only thè right digit. 

The carry bit is 1 if thè sum of thè input bits and previous carry bit is greater than 
1 . This happens when any two of thè bits are 1 : 

c k+1 = (or ( and a k b k ) (and a k c k ) (and b k c k )) 

As with elementary school decimai addition, we start with co = 0, and proceed 
through all thè bits from right to left. 

We can propagate thè equations through thè steps to find a logicai equation for 
each result bit in terms of just thè input bits. First, we simplify thè functions for 
thè first result and carry bits based on knowing co — 0: 

>'o = (xor (xor ao ^o) c o) = ( xor no bo ) 

ci = (or (and ao bo) (and ao c o) (and bo co)) = (and ao bo) 

Then, we can derive thè functions for r k and C 2 ' 

r\ - (xor (xor a\ bi) c k ) = (xor (xor a\ bi) (and ao bo)) 
c 2 = (or (and a\ b k ) (and ai c\) (and b\ ci)) 

= (or (and a\ b k ) (and a k (and a 0 b 0 )) (and b\ (and a 0 b 0 ))) 
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As we move left through thè digits, thè terms get increasingly complex. But, 
for any number of digits, we can always find functions for computing thè result 
bits using only logicai functions on thè input bits. Hence, we can implement 
addition for any length binary numbers using only nand functions. 

We can also implement multiplication, subtraction, and division using only nand 
functions. We omit thè details here, but thè essential approach of breaking down 
our elementary school arithmetic algorithms into functions for computing each 
output bit works for all of thè arithmetic operations. 


Exercise 6.7. Adding logically. 

a. What is thè logicai formula for 

b. Without simplification, how many functions will be composed to compute 
thè addition result bit r 4 ? 

c. [★] Is it possible to compute r 4 with fewer logicai functions? 

Exercise 6.8. Show how to compute thè result bits for binary multiplication of 
two 2-bit inputs using only logicai functions. 

Exercise 6.9. [*] Show how to compute thè result bits for binary multiplication 
of two inputs of any length using only logicai functions. 



6.3 Modeling Computing 

By composing thè logie functions, we could build a wine computer to perform 
any Boolean function. And, we can perform any discrete arithmetic function 
using only Boolean functions. For a useful computer, though, we need pro- 
grammability. We would like to be able to make thè inputs to thè machine de- 
scritte thè logicai functions that it should perform, rather than having to build 
a new machine for each desired function. We could, in theory, construct such a 
machine using wine, but it would be awfully complicated. Instead, we consider 
programmable computing machines abstractly. 

Recali in Chapter 1, we defìned a computer as a machine that can: 

1. Accept input. 

2. Execute a mechanical procedure. 

3. Produce output. 

So, our model of a computer needs to model these three things. 

Modeling input. In reai computers, input comes in many forms: typing on a 
keyboard, moving a mouse, packets coming in from thè network, an accelerom- 
eter in thè device, etc. 


For our model, we want to keep things as simple as possible, though. From a 
computational standpoint, it doesn’t really matter how thè input is collected. We 
can represent any discrete input with a sequence of bits. Input devices like key- 
boards are clearly discrete: there are a finite number of keys, and each key could 
be assigned a unique number. Input from a pointing device like a mouse could 
be continuous, but we can always identify some minimum detected movement 
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distance, and record thè mouse movements as discrete numbers of move units 
and directions. Richer input devices like a camera or microphone can also pro- 
duce discrete output by discretizing thè input using a process similar to thè im- 
age Storage in Chapter 1. So, thè information produced by any input device can 
be represented by a sequence of bits. 

For reai input devices, thè time an event occurs is often cruciai. When playing a 
video game, it does not just matter that thè mouse button was clicked, it matters 
a great deal when thè click occurs. How can we model inputs where time matters 
using just our simple sequence of bits? 

One way would be to divide time into discrete quanta and encode thè input as 
zero or one events in each quanta. A more efficient way would be to add a times- 
tamp to each input. The timestamps are just numbers (e.g., thè number of mil- 
liseconds since thè start time), so can be written down just as sequences of bits. 

Thus, we can model a wide range of complex input devices with just a finite 
sequence of bits. The input must be finite, since our model computer needs all 
thè input before it starts processing. This means our model is not a good model 
for computations where thè input is infinite, such as a web server intended to 
keep running and processing new inputs (e.g., requests for a web page) forever. 
In practice, though, this isn’t usually a big problem since we can make thè input 
finite by limiting thè time thè server is running in thè model. 

A finite sequence of bits can be modeled using a long, narrow, tape that is di- 
vided into squares, where each square contains one bit of thè input. 

Modeling output. Output from computers effects thè physical world in lots of 
very complex ways: displaying images on a screen, printing text on a printer, 
sending an encoded web page over a network, sending an electrical signal to an 
anti-lock brake to increase thè braking pressure, etc. 

We don’t attempt to model thè physical impact of computer outputs; that would 
be far too complicated, but it is also one step beyond modeling thè computa- 
tion itself. Instead, we consider just thè information content of thè output. The 
information in a picture is thè same whether it is presented as a sequence of bits 
or an image projected on a screen, its just less pleasant to look at as a sequence 
of bits. So, we can model thè output just like we modeled thè input: a sequence 
of bits written on a tape divided into squares. 

Modeling processing. Our processing model should be able to model every 
possible mechanical procedure since we want to model a universal computer, 
but should be as simple as possible. 

One thing our model computer needs is a way to keep track of what it is do- 
ing. We can think of this like scratch paperi a human would not be able to do a 
long computation without keeping track of intermediate values on scratch pa- 
per, and a computer has thè same need. In Babbage’s Analytical Engine, this 
was called thè store, and divided into a thousand variables, each of which could 
store a fifty decimai digit number. In thè Apollo Guidance Computer, thè work- 
ing memory was divided into banks, each bank holding 1024 words. Each word 
was 15 bits (plus one bit for errar correction). In current 32 -bit processors, such 
as thè x86, memory is divided into pages, each containing 1024 32-bit words. 


The Virtual 
shopping spree was 
afirstfor thè 
President who has a 
reputatimi for beitig 
“technologically 
challenged." But 
White House 
sources insist that 
thè First Shopper 
used his own laptop 
and even “knew 
how to use thè 
mouse." 
BusinessWeek, 

22 December 1999 
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For our model machine, we don’t want to have arbitrary limits on thè amount of 
working Storage. So, we model thè working Storage with an infmitely long tape. 
Like thè input and output tapes, it is divided into squares, and each square can 
contain one Symbol. For our model computer, it is useful to think about having 
an infmitely long tape, but of course, no reai computer has infinite amounts of 
working Storage. We can, however, imagine continuing to add more memory to 
a reai computer as needed until we have enough to solve a given problem, and 
adding more if we need to solve a larger problem. 

Our model now involves separate tapes for input, output, and a working tape. 
We can simplify thè model by using a single tape for all three. At thè beginning of 
thè execution, thè tape contains thè input (which must be finite). As processing 
is done, thè input is read and thè tape is used as thè working tape. Whatever is 
on thè tape and thè end of thè execution is thè output. 

We also need a way for our model machine to interface with thè tape. We imag- 
ine a tape head that contacts a single square on thè tape. On each processing 
step, thè tape head can read thè Symbol in thè current square, write a Symbol in 
thè current square, and move one square either left or right. 

The final thing we need is a way to model actually doing thè processing. In our 
model, this means controlling what thè tape head does: at each step, it needs to 
decide what to write on thè tape, and whether to move left or right, or to finish 
thè execution. 

In early computing machines, processing meant performing one of thè basic 
arithmetic operations (addition, subtraction, multiplication, or division). We 
don’t want to have to model anything as complex as multiplication in our model 
machine, however. The previous section showed how addition and other arith- 
metic operations can be built from simpler logicai operations. To carry out a 
complex operation as a composition of simple operations, we need a way to 
keep track of enough state to know what to do next. The machine state is just 
a number that keeps track of what thè machine is doing. Unlike thè tape, it is 
limited to a finite number. There are two reasons why thè machine state num- 
ber must be finite: first, we need to be able to write down thè program for thè 
machine by explaining what it should do in each state, which would be difficult 
if there were infmitely many States. 

We also need rules to control what thè tape head does. We can think of each 
rule as a mapping from thè current observed state of thè machine to what to 
do next. The input for a rule is thè Symbol in thè current tape square and thè 
current state of thè machine; thè output of each rule is three things: thè Symbol 
to write on thè current tape square, thè direction for thè tape head to move (left, 
right, or halt), and thè new machine state. We can describe thè program for thè 
machine by listing thè rules. For each machine state, we need a rule for each 
possible symbol on thè tape. 

6.3.1 Turing Machines 

This abstract model of a computer was invented by Alan Turing in thè 1930s 
and is known as a Turing Machine. Turing’s model is depicted in Figure 6.4. 
An infinite tape divided into squares is used as thè input, working Storage, and 
output. The tape head can read thè current square on thè tape, write a symbol 
into thè current tape square, and move left or right one position. The tape head 
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Figure 6.4. Turing Machine model. 


keeps track of its internai state, and follows rules matching thè current state and 
current tape square to determine what to do next. 

Turing’s model is by far thè most widely used model for computers today. Tur- 
ing developed this model in 1936, before anything resembling a modern com- 
puter existed. Turing did not develop his model as a model of an automatic 
computer, but instead as a model for what could be done by a human following 
mechanical rules. He devised thè infinite tape to model thè two- dimensionai 
graph paper students use to perform arithmetic. He argued that thè number 
of machine States must be limited by arguing that a human could only keep a 
limited amount of Information in mind at one time. 

Turing’s model is equivalent to thè model we described earlier, but instead of 
using only bits as thè symbols on thè tape, Turing’s model uses members of any 
finite set of symbols, known as thè alphabet of thè tape. Allowing thè tape al- 
phabet to contain any set of symbols instead of just thè two binary digits makes 
it easier to describe a Turing Machine that computes a particular function, but 
does not change thè power of thè model. That means, every computation that 
could be done with a Turing Machine using any alphabet set, could also be done 
by some Turing Machine using only thè binary digits. 

We could show this by describing an algorithm that takes in a description of a 
Turing Machine using an arbitrarily large alphabet, and produces a Turing Ma- 
chine that uses only two symbols to simulate thè input Turing Machine. As we 
saw in Chapter 1, we can map each of thè alphabet symbols to a finite sequence 
of binary digits. 

Mapping thè rules is more complex: since each originai input Symbol is now 
spread over several squares, we need extra States and rules to read thè equiva- 
lent of one originai input. For example, suppose our originai machine uses 16 
alphabet symbols, and we map each Symbol to a 4-bit sequence. If thè originai 
machine used a Symbol X, which we map to thè sequence of bits 1011, we would 
need four States for every state in thè originai machine that has a rule using X as 
input. These four States would read thè 1, 0, 1, 1 from thè tape. The last state 
now corresponds to thè state in thè originai machine when an X is read from thè 
tape. To follow thè rule, we also need to use four States to write thè bit sequence 
corresponding to thè originai write Symbol on thè tape. Then, simulating mov- 
ing one square left or right on thè originai Turing Machine, now requires moving 
four squares, so requires four more States. Hence, we may need 12 States for each 
transition rule of thè originai machine, but can simulate everything it does using 
only two symbols. 
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universa l The Turing Machine model is a universal computing machine. This means every 
computing machine algorithm can be implemented by some Turing Machine. Chapter 12 explores 
more deeply what it means to simulate every possible Turing Machine and ex- 
plores thè set of problems that can be solved by a Turing Machine. 

Any physical machine has a limited amount of memory. If thè machine does 
not have enough space to store a trillion bits, there is no way it can do a com- 
putation whose output would exceed a trillion bits. Nevertheless, thè simplicity 
and robustness of thè Turing Machine model make it a useful way to think about 
computing even if we cannot build a truly universal computing machine. 

Turing’s model has proven to be remarkably robust. Despite being invented 
before anything resembling a modem computer existed, nearly every comput- 
ing machine ever imagined or built can be modeled well using Turing’s simple 
model. The important thing about thè model is that we can simulate any com- 
puter using a Turing Machine. Any step on any computer that operates using 
standard physics and be simulated with a finite number of steps on a Turing 
Machine. This means if we know how many steps it takes to solve some prob- 
lem on a Turing Machine, thè number of steps it takes on any other machine is 
at most some multiple of that number. Hence, if we can reason about thè num- 
ber of steps required for a Turing Machine to solve a given problem, then we can 
make strong and generai claims about thè number of steps it would take any 
standard computer to solve thè problem. We will show this more convincingly 
in Chapter 12, but for now we assert it, and use it to reason about thè cost of 
executing various procedures in thè following chapter. 


Example6.1: Balancing Parentheses 


We define a Turing Machine that solves thè problem of checking parentheses 
are well-balanced. For example, in a Scheme expression, every opening left 
parenthesis must have a corresponding closing right parenthesis. For example, 
(()(()))() is well-balanced, but (()))(() is not. Our goal is to design a Turing 
Machine that takes as input a string of parentheses (with a # at thè beginning 
and end to mark thè endpoints) and produces as output a 1 on thè tape if thè 
input string is well-balanced, and a 0 otherwise. For this problem, thè output is 
what is written in thè square under thè tape head; it doesn’t matter what is left 
on thè rest of thè tape. 

Our strategy is to fìnd matching pairs of parentheses and cross them out by writ- 
ing an X on thè tape in place of thè parenthesis. If all thè parentheses are crossed 
out at thè end, thè input was well-balanced, so thè machine writes a 1 as its out- 
put and halts. If not, thè input was not well-balanced, and thè machine writes 
a 0 as its output and halts. The trick to thè matching is that a closing parenthe- 
sis always matches thè frrst open parenthesis found moving to thè left from thè 
closing parenthesis. The pian for thè machine is to move thè tape head to thè 
right (without changing thè input) until a closing parenthesis is found. Cross 
out that closing parenthesis by replacing it with an X, and move to thè left un- 
til an open parenthesis is found. This matches thè closing parenthesis, so it is 
replaced with an X. Then, continue to thè right searching for thè next closing 
parenthesis. If thè end of thè tape (marked with a #) is found, check thè tape has 
no remaining open parenthesis. 

We need three internai States: LookForClosing, in which thè machine contin- 
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ues to thè right until it fìnds a closing parenthesis (this is thè start state); Look- 
ForOpen, in which thè machine continues to thè left until it finds thè balancing 
open parenthesis; and CheckTape, in which thè machine checks there are no un- 
balanced open parentheses on thè tape starting from thè right end of thè tape 
and moving towards thè left end. The full rules are shown in Figure 6.5. 


State 

Read 

Next State 

Write 

Move 


LookForClosing 

) 

LookForOpen 

X 

<- 

Found closing. 

LookForClosing 

( 

LookForClosing 

( 

— > 

Keep looking. 

LookForClosing 

X 

LookForClosing 

X 

— > 

Keep looking. 

LookForClosing 

# 

CheckTape 

# 


End oftape. 

LookForOpen 

) 

- 

X 

Error 

Shouldn't happen. 

LookForOpen 

( 

LookForClosing 

X 

— > 

Found open. 

LookForOpen 

X 

LookForOpen 

X 

t- 

Keep looking. 

LookForOpen 

# 

- 

0 

Halt 

Reached beginning. 

CheckTape 

) 

- 

0 

Error 

Shouldn’t happen. 

CheckTape 

( 

- 

0 

Halt 

Unbalanced open. 

CheckTape 

X 

CheckTape 

X 

«- 

Keep checking. 

CheckTape 

# 

- 

1 

Halt 

Finished checking. 


Figure 6.5. Rules for checking balanced parentheses Turing Machine. 


Another way to depict a Turing Machine is to show thè States and rules graphi- 
cally. Each state is a node in thè graph. For each rule, we draw an edge on thè 
graph between thè starting state and thè next state, and label thè edge with thè 
read and write tape symbols (separated by a /), and move direction. 

Figure 6.6 shows thè same Turing Machine as a state graph. When reading a 
Symbol in a given state produces an error (such as when a ) is encountered in 
thè LookForOpen state) , it is not necessary to draw an edge on thè graph. If there 
is no outgoing edge for thè current read symbol for thè current state in thè state 
graph, execution terminates with an error. 



#/#,<- 


^ #/l,Halt 
(/0,Haìt 


V 




Halt 




A 


Figure 6.6. Checking parentheses Turing Machine. 
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Bombe 

Rebuilt at Bletchley Park 


Exercise 6.10. Follow thè rules to simulate thè checking parentheses Turing 
Machine on each input (assume thè beginning and end of thè input are marked 
with a #): 

a. ) 

b. () 

c. empty input 

d. (()(()))() 

e. (()))(() 

Exercise 6.1 1. [*] Design a Turing Machine for adding two arbitrary-length bi- 
nary numbers. The input is of thè form a n _\ . . . a\a o + b m _\ . . . b\bo (with # mark- 
ers at both ends) where each a k and b k is either 0 or 1. The output tape should 
contain bits that represent thè sum of thè two inputs. 


Profile: Alan Turing 


Alan Turing was born in London in 1912, and developed his computing model 
while at Cambridge in thè 1930s. He developed thè model to solve a famous 
problem posed by David Hilbert in 1928. The problem, known as thè Entschei- 
dungsproblem (German for “decision problem”) asked for an algorithm that could 
determine thè truth or falsehood of a mathematical statement. To solve thè 
problem, Turing first needed a formai model of an algorithm. For this, he in- 
vented thè Turing Machine model described above, and defined an algorithm 
as any Turing Machine that is guaranteed to eventually halt on any input. With 
thè model, Turing was able to show that there are some problems that cannot 
be solved by any algorithm. We return to this in Chapter 12 and explain Turing’s 
proof and examples of problems that cannot be solved. 

After publishing his solution to thè Entscheidungsproblem in 1936, Turing went 
to Princeton and studied with Alonzo Church (inventor of thè Lambda calcu- 
lus, on which Scheme is based). With thè start of World War II, Turing joined 
thè highly secret British effort to break Nazi codes at Bletchley Park. Turing 
was instrumentai in breaking thè Enigma code which was used by thè Nazi’s 
to communicate with field units and submarines. Turing designed an electro- 
mechanical machine known as a bombe for searching possible keys to decrypt 
Enigma- encrypted messages. The machines used logicai operations to search 
thè possible rotor settings on thè Enigma to find thè settings that were most 
likely to have generated an intercepted encrypted message. Bletchley Park was 
able to break thousands of Enigma messages during thè war. The Allies used thè 
knowledge gained from them to avoid Nazi submarines and gain a tremendous 
tactical advantage. 

After thè war, Turing continued to make both practical and theoretical contri- 
butions to computer Science. Among other things, he worked on designing 
general-purpose computing machines and published a paper ( Intelligent Ma- 
chinery) speculating on thè ability of computers to exhibit intelligence. Turing 
introduced a test for machine intelligence (now known as thè Turing Test) based 
on a machines ability to impersonate a human and speculated that machines 




Chapter6. Machines 


123 


would be able to pass thè test within 50 years (that is, by thè year 2000) . Tur- 
ing also studied morphogenesis (how biological systems grow) including why 
Fibonacci numbers appear so often in plants. 

In 1952, Turing’s house was broken into, and Turing reported thè crime to thè 
police. The investigation revealed that Turing was a homosexual, which at thè 
time was considered a crime in Britain. Turing did not attempt to hide his homo- 
sexuality, and was convicted and given a choice between serving time in prison 
and taking hormone treatments. He accepted thè treatments, and has his se- 
curity clearance revoked. In 1954, at thè age of 41, Turing was found dead in 
an apparent suicide, with a cynide-laced partially-eaten appiè next to him. The 
codebreaking effort at Bletchley Park was kept secret for many years after thè 
war (Turing’s report on Enigma was not declassifred until 1996), so Turing never 
received public recognition for his contributions to thè war effort. In September 
2009, instigated by an on-line petition, British Prime Minister Gordon Brown 
issued an apology for how thè British government treated Alan Turing. 


6.4 Summary 

The power of computers comes from their programmability. Universal comput- 
ers can be programmed to execute any algorithm. The Turing Machine model 
provides a simple, abstract, model of a computing machine. Every algorithm 
can be implemented as a Turing Machine, and a Turing Machine can simulate 
any other reasonable computer. 

As thè fìrst computer programmer, Ada deserves thè last word: 

By thè word operation, we meati any process which alters thè mutuai re- 
lation oftwo or more things, be this relation ofwhat kind it may. This is 
thè most generai definition, and would include all subjects in thè universe. 

In abstract mathematics, ofcourse operations alter those particular rela- 
tions which are involved in thè considerations ofnumber and space, and 
thè results of operations are those peculiar results which correspond to thè 
nature of thè subjects of operation. But thè Science of operations, as de- 
rived from mathematics more especially, is a Science ofitself, and has its 
own abstract truth and value; just as logie has its own peculiar truth and 
value, independently ofthe subjects to which we may apply its reasonings 
and processes 

The operating mechanism can even be thrown into action independently 
of any object to operate upon (although ofcourse no result could then be 
developed). Again, it might act upon other things besides number, were 
objects found whose mutuai fundamental relations could be expressed by 
those ofthe abstract Science of operations, and which should be also sus- 
ceptible of adaptations to thè action ofthe operating notation and mech- 
anism ofthe engine. Supposing, for instance, that thè fundamental rela- 
tions ofpitched sounds in thè Science ofharmony and of musical composi- 
tion were susceptible ofsuch expression and adaptations, thè engine might 
compose elaborate and scientific pieces of music ofany degree ofcomplex- 
ity or extent. 

Ada, Countess of Lovelace, Sketch of The Analytical Engine, 1843 
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6.4. Summary 


online library 



Cost 


A LISP programmer knows thè value ofeverything, but thè cost ofnothing. 

Alan Perlis 

I told my dad that someday I'd have a computer that I could write programs on. He said that would 
cost as much as a house. I said, “Well, then I'm going to live in an apartment." 

Steve Wozniak 

This chapter develops tools for reasoning about thè cost of evaluating a given 
expression. Predicting thè cost of executing a procedure has practical value (for 
example, we can estimate how much computing power is needed to solve a par- 
ticular problem or decide between two possible implementations), but also pro- 
vides deep insights into thè nature of procedures and problems. 

The most commonly used cost metric is time. Other measures of cost include 
thè amount of memory needed and thè amount of energy consumed. Indirectly, 
these costs can often be translated into money: thè rate of transactions a Service 
can support, or thè price of thè computer needed to solve a problem. 


7.1 Empirical Measurements 

We can measure thè cost of evaluating a given expression empirically. If we are 
primarily concerned with time, we could just use a stopwatch to measure thè 
evaluation time. For more accurate results, we use thè built-in ( time Expression) 
special form. 1 Evaluating ( time Expression ) produces thè value of thè input ex- 
pression, but also prints out thè time required to evaluate thè expression (shown 
in our examples using slanted font). It prints out three time values: 

cpu time 

The time in milliseconds thè processor ran to evaluate thè expression. CPU 
is an abbreviation for “centrai processing unit”, thè computer’s main pro- 
cessor. 

reai time 

The actual time in milliseconds it took to evaluate thè expression. Since 
other processes may be running on thè computer while this expression 
is evaluated, thè reai time may be longer than thè CPU time, which only 
counts thè time thè processor was working on evaluating this expression. 

1 The time construct must bea special form, since thè expression is not evaluated before entering 
time as it would be with thè normal application rule. If it were evaluated normally, there would be 
no way to time how long it takes to evaluate, since it would have already been evaluated before time 
is applied. 
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gc time 

The time in milliseconds thè interpreter spent on garbage collection to eval- 
uate thè expression. Garbage collection is used to reclaim memory that is 
storing data that will never be used again. 

For example, using thè definitions from Chapter 5, 

( time ( solve- pegboa rd ( board- remove- peg ( make-board 5) 

{make-positìon 1 1)))) 

prints: cpu time: 141 797 reai time: 152063 gc time: 765. The reai time is 152 seconds, 
meaning this evaluation took just over two and a half minutes. Of this time, thè 
evaluation was using thè CPU for 142 seconds, and thè garbage collector ran for 
less than one second. 

Here are two more examples: 

> ( time ( car ( list-append ( intsto 1000) ( intsto 100)))) 
cpu time: 531 reai time: 531 gc time: 62 

1 

> ( time ( car { list-append {intsto 1000) (misto 100)))) 
cpu time: 609 reai time: 609 gc time: 0 

1 


There’s no seme in 
being precise when 
you don’t even know 
whatyou’re talking 
about. 
John von Neumann 


The two expressions evaluated are identical, but thè reported time varies. Even 
on thè same computer, thè time needed to evaluate thè same expression varies. 
Many properties unrelated to our expression (such as where things happen to 
be stored in memory) impact thè actual time needed for any particular evalua- 
tion. Hence, it is dangerous to draw conclusions about which procedure is faster 
based on a few timings. 

Another limitation of this way of measuring cost is it only works if we wait for thè 
evaluation to complete. If we try an evaluation and it has not hnished after an 
hour, say, we have no idea if thè actual time to finish thè evaluation is sixty-one 
minutes or a quintillion years. We could wait another minute, but if it stili hasn’t 
finished we don’t know if thè execution time is sixty-two minutes or a quintillion 
years. The techniques we develop allow us to predict thè time an evaluation 
needs without waiting for it to execute. 

Finally, measuring thè time of a particular application of a procedure does not 
provide much insight into how long it will take to apply thè procedure to differ- 
ent inputs. We would like to understand how thè evaluation time scales with thè 
size of thè inputs so we can understand which inputs thè procedure can sensibly 
be applied to, and can choose thè best procedure to use for different situations. 
The next section introduces mathematical tools that are helpful for capturing 
how cost scales with input size. 


Exercise 7.1. Suppose you are defrning a procedure that needs to append two 
lists, one short list, short and one very long list, long, but thè order of elements 
in thè resulting list does not matter. Is it better to use ( list-append short long ) or 
{list-append long short)? (A good answer will involve both experimental results 
and an analytical explanation.) 
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Exploration 7.1: Multiplying Like Rabbits 


Filius Bonacci was an Italian monlc and mathematician in thè 12th century. He 
published a hook, Liber Abbaci, on how to calculate with decimai numbers that 
introduced Hindu -Arabie numbers to Europe (replacing Roman numbers) along 
with many of thè algorithms for doing arithmetic we learn in elementary school. 
It also included thè problem for which Fibonacci numbers are named: 2 

A pair of newly-born male and female rabbits are put in a field. Rabbits 
mate at thè age ofone month and after that procreate every month, so thè 
female rabbit produces a new pair of rabbits at thè end ofits second month. 
Assume rabbits never die and that each female rabbit produces one new 
pair (one male, one female) every month from her second month on. How 
many pairs will there be in oneyear? 



We can debne a function that gives thè number of pairs of rabbits at thè begin- 
ning of thè n th month as: 


Filius Bonacci 


Fibonacci(n) 


1 

1 

Fibonacciln — 1) + Fibonacci(n — 2) 


n — 1 
n = 2 
n > 1 


The third case follows from Bonacci’s assumptions: all thè rabbits alive at thè 
beginning of thè previous month are stili alive (thè Fibonacci(n — 1) term), and 
all thè rabbits that are at least two months old reproduce (thè Fibonacci(n — 2) 
term) . 

The sequence produced is known as thè Fibonacci sequence: 

1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, . . . 


After thè first two ls, each number in thè sequence is thè sum of thè previous 
two numbers. Fibonacci numbers occur frequently in nature, such as thè ar- 
rangement of florets in thè sunflower (34 spirals in one direction and 55 in thè 
other) or thè number of petals in common plants (typically 1, 2, 3, 5, 8, 13, 21, or 
34), hence thè rarity of thè four-leaf dover. 

Translating thè definition of thè Fibonacci function into a Scheme procedure is 
straightforward; we combine thè two base cases using thè or special form: 

(define {Jìbo n ) 

(if (or (= n 1) (= n2)) 1 

(+ {flbo (-ni)) [flbo (- n 2))))) 



Applying/ìbo to small inputs works fine: 


> ( time Ifibo 10)) 

epu time: 0 reai time: 0 gc time: 0 
55 

> ( time ifibo 30)) 

epu time: 2156 reai time: 2187 gc time: 0 
832040 

2 Although thè sequence is named for Bonacci, it was probably not invented by him. The se- 
quence was already known to Indian mathematicians with whom Bonacci studied. 
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But when we try to determine thè number of rabbits in five years by computing 
{flbo 60), our interpre ter just hangs without producing a value. 

The flbo procedure is defined in a way that guarantees it eventually completes 
when applied to a non-negative whole number: each recursive cali reduces thè 
input by 1 or 2, so both recursive calls get closer to thè base case. Hence, we 
always make progress and must eventually reach thè base case, unwind thè re- 
cursive applications, and produce a value. To understand why thè evaluation of 
[flbo 60) did not finish in our interpreter, we need to consider how much work is 
required to evaluate thè expression. 

To evaluate ( flbo 60), thè interpreter follows thè if expressions to thè recursive 
case, where it needs to evaluate (+ ( flbo 59) ( flbo 58)). To evaluate ( flbo 59), it 
needs to evaluate ( flbo 58) again and also evaluate ( flbo 57). To evaluate [flbo 58) 
(which needs to be done twice), it needs to evaluate ( flbo 57) and ( flbo 56). So, 
there is one evaluation of ( flbo 60), one evaluation of ( flbo 59), two evaluations 
of ( flbo 58), and three evaluations of ( flbo 57). 

The total number of evaluations of thè flbo procedure for each input is itself 
thè Fibonacci sequence! To understand why, consider thè evaluation tree for 
( flbo 4) shown in Figure 7.1. The only direct number values are thè 1 values that 
result from evaluations of either ( flbo 1) or ( flbo 2). Hence, thè number of 1 val- 
ues must be thè value of thè final result, which just sums all these numbers. 
For ( flbo 4), there are 5 leaf applications, and 3 more inner applications, for 8 
(= Fibonaccfl 5)) total recursive applications. The number of evaluations of ap- 
plications of flbo needed to evaluate {flbo 60) is thè 61st Fibonacci number — 
2,504,730,781,961 — over two and a half trillion applications of flbo\ 


( flbo 5) 



{flbo 4) {flbo 3) 



{flbo 2) {flbo 2) {flbo 1 ) 

1 1 1 


Figure 7.1. Evaluation of flbo procedure. 


{flbo 3) 

( flbo 2) {flbo 1 ) 


Although our flbo definition is correct, it is ridiculously inefficient and only fin- 
ishes for input numbers below about 40. It involves a tremendous amount of 
duplicated work: for thè {flbo 60) example, there are two evaluations of {flbo 58) 
and over a trillion evaluations of {flbo 1 ) and {flbo 2). 

We can avoid this duplicated effort by building up to thè answer starting from 
thè base cases. This is more like thè way a human would determine thè numbers 
in thè Fibonacci sequence: we find thè next number by adding thè previous two 
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numbers, and stop once we have reached thè number we want. 

Th e fast- fibo procedure computes thè n th Fibonacci number, but avoids thè du- 
plicate effort by computing thè results building up from thè first two Fibonacci 
numbers, instead of working backwards. 

(define ( fast-flbo n) 

(define ( flbo-iter a b left ) 

(if [< — left 0) b 

[flbo-iter b[+ ab) (- left 1)))) 

[flbo-iter 11 [— n 2))) 

This is a form ofwhat is known as dynamic programming. The definition is stili 
recursive, but unlike thè originai definition thè problem is broken down differ- 
ently. Instead of breaking thè problem down into a slightly smaller instance of 
thè originai problem, with dynamic programming we build up from thè base 
case to thè desired solution. In thè case of Fibonacci, thè fast-flbo procedure 
builds up from thè two base cases until reaching thè desired answer. The addi- 
tional complexity is we need to keep track of when to stop; we do this using thè 
left parameter. 

The helper procedure, flbo-iter (short for iteration), takes three parameters: a 
is thè value of thè previous-previous Fibonacci number, b is thè value of thè 
previous Fibonacci number, and left is thè number of iterations needed be- 
fore reaching thè target. The initial cali to flbo-iter passes in 1 as a (thè value 
of Fibonacci( 1)), and 1 as b (thè value of Fibonacci( 2)), and (— n 2) as left (we 
have n — 2 more iterations to do to reach thè target, since thè first two Fibonacci 
numbers were passed in as a and b we are now working on Fibonaccfl 2)) . Each 
recursive cali to flbo-iter reduces thè value passed in as left by one, and advances 
thè values of a and b to thè next numbers in thè Fibonacci sequence. 

The fast-flbo procedure produces thè same output values as thè originai fibo 
procedure, but requires far less work to do so. The number of applications of 
flbo-iter needed to evaluate [fast-flbo 60) is now only 59. The value passed in as 
left for thè first application of flbo-iter is 58, and each recursive cali reduces thè 
value of left by one until thè zero case is reached. This allows us to compute thè 
expected number of rabbits in 5 years is 1548008755920 (over 1.5 Trillion) 3 . 


7.2 Orders of Growth 

As illustrated by thè Fibonacci exploration, thè same problem can be solved 
by procedures that require vastly different resources. The important question 
in understanding thè resources required to evaluate a procedure application is 
how thè required resources scale with thè size ofthe input. For small inputs, both 
Fibonacci procedures work using with minimal resources. For large inputs, thè 
first Fibonacci procedure never finishes, but thè fast Fibonacci procedure fin- 
ishes effectively instantly. 

In this section, we introduce three functions computer scientists use to capture 


3 Perhaps Bonacci’s assumptions are not a good model for actual rabbit procreation. This result 
suggests that in about 10 years thè mass of all thè rabbits produced from thè initial pair will exceed 
thè mass of thè Earth, which, although scary, seems unlikely! 


dynamic 
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thè important properties of how resources required grow with input size. Each 
function takes as input a function, and produces as output a set of functions: 

0(f) (“big oh”) 

The set of functions that grow nofaster than/ grows. 

0(/) (theta) 

The set of functions that grow asfast as / grows. 
n(/) (omega) 

The set of functions that grow no slower than / grows. 


Remember that 
accumulateci 
knowledge, like 
accumulated 
capitai, increasesat 
compound interest: 

but it differsfrom 
thè accumulation of 
capitai in this ; that 
thè increase of 
knowledge produces 
a more rapid rate of 
progress, whilstthe 
accumulation of 
capitai leads to a 
lower rate of 
interest. Capital 
thus checks its own 
accumulation: 
knowledge thus 
accelerates its own 
advance. Each 
generation, 
therefore, to deserve 
comparison with its 
predecessor, is 
bound to add much 
more largely to thè 
common stock than 
that which it 
immediately 
succeeds. 
Charles Babbage, 1851 


These functions capture thè asymptotic behavior of functions, that is, how they 
behave as thè inputs get arbitrarily large. To understand how thè time required 
to evaluate a procedure increases as thè inputs to that procedure increase, we 
need to know thè asymptotic behavior of a function that takes thè size of input 
to thè target procedure as its input and outputs thè number of steps to evaluate 
thè target procedure on that input. 

Figure 7.2 depicts thè sets O, 0, 0 for some function /. Next, we define each 
function and provide some examples. Section 7.3 illustrates how to analyze thè 
time required to evaluate applications of procedures using these notations. 


Functions that 
grow faster 
than / 



Functions that grow 
as fast as/ 


Figure 7.2. Visualization of thè sets O(f), fi(/), and 0(/). 


7.2.1 Big O 

The hrst notation we introduce is O, pronounced “big oh”. The O function takes 
as input a function, and produces as output thè set of all functions that grow no 
faster than thè input function. The set 0(/) is thè set of all functions that grow 
as fast as, or slower than, / grows. In Figure 7.2, thè O(f) set is represented by 
everything inside thè outer circle. 


To define thè meaning of O precisely, we need to consider what it means for a 
function to grow. We want to capture how thè output of thè function increases 
as thè input to thè function increases. First, we consider a few examples; then 
we provide a formai definition of O. 
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f(n ) — n + 12 and g(n) — n — 7 

No matter what n value we use, thè value of f(n) is greater than thè value of 
g(n). This doesn’t matter for thè growth rates, though. What matters is how 
thè difference between g(n ) and f(n) changes as thè input values increase. 
No matter what values we choose for n\ and n 2 , we knowythj) — f(n\) = 
g(n 2 ) - f{n 2 ) — -19. Thus, thè growth rates of / and g are identical and 
n — 7 is in thè set 0(n + 12), and n + 12 is in thè set 0(n — 7). 

f(n) — In and g(n) — 3 n 

The difference between g(n) and f(n) is n. This difference increases as thè 
input value n increases, but it increases by thè same amount as n increases. 
So, thè growth rate as n increases is ^ = 1. The value of 2 n is always within 
a Constant multiple of 3 n, so they grow asymptotically at thè same rate. 
Hence, 2 n is in thè set O (3n ) and 3 n is in thè set O (2n ) . x 

f(n) — n and g{n) — n 2 

The difference between g(n) and f(n) is n 2 — n — n(n — 1). The growth rate 
as n increases is — n — 1. The value of n — 1 increases as n increases, 

so g grows faster than /. This means n 2 is not in 0(n) since n 2 grows faster 
than n. The function n is in Oin 2 ) since n grows slower than n 2 grows. 

f(n) — Fibonacci(n ) and g(n) — n 

The Fibonacci function grows very rapidly. The value of Fibonacci(n + 2) 
is more than doublé thè value of Fibonacci(n) since 


Fibonacci(n + 2) = Fibonacci(n + 1) + Fibonacci(n) 


and Fibonacci(n + 1) > Fibonacci(n). The rate of increase is multiplicative, 
and must be at least a factor of y/2 g 1.414 (since increasing by one twice 
more than doubles thè value). (In fact, thè rate of increase is a factor of 
cp — (1 + \/5) /2 1.618, also known as thè “golden ratio”. This is a rather 

remarkable result, but explaining why is beyond thè scope of this hook.) 
This is much faster than thè growth rate of n, which increases by one when 
we increase n by one. So, n is in thè set 0(Fibonacci(n)), but Fibonacci(n) 
is not in thè set 0(n). 

Some of thè example functions are plotted in Figure 7.3. The O notation reveals 
thè asymptotic behavior of functions. The functions plotted are thè same in 
both graphs, but thè scale of thè horizontal axis is different. In thè fìrst graph, thè 
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rightmost value of n 2 is greatest; for higher input values, thè value of Fibonacci{n) 
is greatest. In thè second graph, thè values of Fibonacci(n) for input values up to 
20 are so large that thè other functions appear as nearly fiat lines on thè graph. 

Definition of O. The function g is a member of thè set O(f) if and only if there 
exist positive constants c and iiq such that, for all values n > no, 

g(n) < cf(n). 

We can showg is in O(f) using thè definition of O(f) by choosing positive con- 
stants for thè values of c and no, and showing that thè property g(n ) < cf(n) 
holds for all values n > n 0 . To showy is not in 0{f), we need to explain how, for 
any choices of c and no, we can find values of n that are greater than iiq such that 
g(n) < cf(n) does nothold. 


Example 7.1: O Examples 


We now show thè claimed properties are true using thè formai definition. 
n — 7 is in 0(n + 12) 

Choose c — 1 and no — 1. Then, we need to show« — 7 < l(n + 12) for all 
values n > 1. This is true, since n — 7 > n + 12 for all values n. 

n + 12 is in 0(n — 7) 

Choose c = 2 and no — 26. Then, we need to show n + 12 < 2 (n — 7) for all 
values n > 26. The equation simplifies to n + 12 < 2n — 14, which simplifìes 
to 26 < n. This is trivially true for all values n > 26. 

2 n is in 0(3n) 

Choose c = 1 and no — 1. Then, 2 n < 3 n for all values n > 1. 

3 n is in 0(2n) 

Choose c = 2 and «o = 1. Then, 3 n < 2(2 n) simplifìes to n < 4/3 n which is 
true for all values n > 1. 

n is in 0(n 2 ) 

Choose c — 1 and no — 1. Then n < n 2 for all values n > 1. 
n 2 is not in 0(n) 

We need to show that no matter what values are chosen for c and no, there 
are values of n > no such that thè inequality n 2 < cn does not hold. For any 
value of c, we can make n 2 > cn by choosing n > c. 

n is in 0(Fibonacci{n )) 

Choose c — 1 and no — 3. Then n < Fibonacci(n) for all values n > no- 
Fibonacci(n) is not in 0(n — 2) 

No matter what values are chosen for c and no, there are values of n > no 
such that Fibonacci(n) > c(n). We know Fibonacci(\2) — 144, and, from 
thè discussion above, that: 

Fibonacci(n + 2) > 2* Fibonacci(n ) 

This means, for n > 12, we know Fibonacci(n) > n 2 . So, no matter what 
value is chosen for c, we can choose n — c. Then, we need to show 

Fibonacci(n ) > n(n) 

The right side simplifìes to n 2 . For n > 12, we know Fibonacci(n ) > ri 2 . 
Hence, we can always choose an n that contradicts thè Fibonacci(n) < cn 
inequality by choosing an n that is greater than n 0 , 12, and c. 
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For all of thè examples where g is in O (/) , there are many acceptable choices for 
c and «o- F°r thè given c values, we can always use a higher no value than thè 
selected value. It only matters that there is some finite, positive Constant we can 
choose for no, such that thè required inequality, gin) < c/(n) holds for all values 
n > n 0 . Hence, our proofs work equally well with higher values for n 0 than we 
selected. Similarly, we could always choose higher c values with thè same n 0 
values. The key is just to pick any appropriate values for c and no, and show thè 
inequality holds for all values n > no- 

Proving that a function is not in 0(/) is usually tougher. The key to these proofs 
is that thè value of n that invalidates thè inequality is selected after thè values of 
c and no are chosen. One way to think of this is as a game between two adver- 
saries. The first player picks c and no, and thè second player picks n. To show thè 
property that g is not in 0(/), we need to show that no matter what values thè 
first player picks for c and n 0 , thè second player can always find a value n that is 
greater than no such that g(n ) > cf(n). 

Exercise 7.2. For each of thè g functions below, answer whether or not g is in thè 
set O(n). Your answer should include a proof. Ify is in O(n) you should identify 
values of c and hq that can be selected to make thè necessary inequality hold. 
If g is not in O(n) you should argue convincingly that no matter what values 
are chosen for c and no there are values of n > no such thè inequality in thè 
definition of O does not hold. 

a. g(n) — n + 5 

b. g(n) = .01 n 

c. g(n) — 150n + fn 

d. g(n) — n 15 

e. g(n) — n\ 

Exercise 7.3. [*] Given / is some function in 0(h), and g is some function not in 
0(h), which of thè following must always be true: 

a. For all positive integers ni, f(m) < g(m). 

b. For some positive integer m, f(m ) < g(m). 

c. For some positive integer m 0 , and all positive integers m > m 0 , 

f(m) <g(m). 


7.2.2 Omega 

The set Q(/) (omega) is thè set of functions that grow no slower than / grows. 
So, a function g is in Q(/) if g grows as fast as / or faster. Constrast this with 
0(/), thè set of all functions that grow no faster than / grows. In Figure 7.2, 
fi(/) is thè set of all functions outside thè darker circle. 

The formai definition of Q(/) is nearly identical to thè definition of 0(/): thè 
only difference is thè < comparison is changed to >. 

Definition of Q(/). The function g is a member of thè set fi(/) if and only if 
there exist positive constants c and uq such that, for all values n > no, 


g(n) > cf(n). 
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Example 7.2: O Examples 


We repeat selected examples from thè previous section with fi instead of O. The 
strategy is similar: we show g is in Q(/) using thè definition of Q( /) by choos- 
ing positive constants for thè values of c and no, and showing that thè property 
g(n) > cf(n) holds for all values n > no. To showy is not in fi(/), we need to 
explain how, for any choices of c and no, we can hnd a choice for n > no such 
thaty(n) < cf(n). 

n — 7 is in Cì(n + 12) 

Choose c — j an d no — 26. Then, we need to show n — 7 > \(n + 12) for 
all values n > 26. This is true, since thè inequality simplifies | > 13 which 
holds for all values n > 26. 

2 n is in Q(3n) 

Choose c — j and no — 1. Then, 2n > j (3 n) simplifies to n > 0 which holds 
for all values n > 1. 

n is not in Cì(n 2 ) 

Whatever values are chosen for c and n 0 , we can choose n > n 0 such that 
n > cn 2 does not hold. Choose n >1 (note that c must be less than 1 for 
thè inequality to hold for any positive n, so if c is not less than 1 we can just 
choose n > 2). Then, thè right side of thè inequality cn 2 will be greater than 
n, and thè needed inequality n > cn 2 does not hold. 

n is not in CÌ(Fibonacci(n)) 

No matter what values are chosen for c and n 0 , we can choose n > n 0 such 
that n > Fibonacci(n) does not hold. The value of Fibonacci(n) more than 
doubles every time n is increased by 2 (see Section 7.2.1), but thè value 
of c{n) only increases by 2c. Hence, if we keep increasing n, eventually 
Fibonacci(n + 1) > c(n - 2) for any choice of c. 


Exercise 7.4. Repeat Exercise 7.2 using Q instead of O. 

Exercise 7.5. For each part, identify a function g that satisfies thè property. 

a. g is in 0(n 2 ) but not in Cl(n 2 ). 

b. g is not in 0(n 2 ) but is in Q(« 2 ). 

c. g is in both 0(n 2 ) and Q(« 2 ). 

7.2.3 Theta 

The function 0(/) denotes thè set of functions that grow at thè same rate as /. 
It is thè intersection of thè sets O(f) and O (/). Hence, a function g is in ©(/) if 
and only if g is in 0(/) and g is in Q(/). In Figure 7.2, 0(/) is thè ring between 
thè outer and inner circles. 

An alternate definition combines thè inequalities for O and O: 

Definition of 0(/). The function g is a member of thè set 0(/) if any only if 
there exist positive constants ci, ci, and no such that, for all values n > no, 


cif{n) > g(n) > c 2 f(n). 
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Ifg(n) is in 0(/(n)), then thè sets ©(/(«)) and0(g(«)) are identical. Ify(n) e 
0 (/(n) ) then g and / grow at thè same rate, 


Example 7.3: 0 Examples 


Determining membership in 0(/) is simple once we know membership in 0(f) 
and n(/). 

n — 7 is in 0(n + 12) 

Since « — 7is inO(« + 12) and n — 7 is inO (n + 12) weknown — 7 is in 0(n + 
12). Intuitively, n — 7 increases at thè same rate as n + 12, since adding one 
to n adds one to both function outputs. We can also show this using thè 
definitionof0(/): chooseci = 1, c-i — and no = 38. 

In is in 0(3 n) 

2 n is in 0(3n) and in Cl(3n). Choose c\ — 1,C2 — 5 , and no — 1. 
n is not in @(n 2 ) 

n is not in n(« 2 ). Intuitively, n grows slower than n 2 since increasing n by 
one always increases thè value of thè first function, n, by one, but increases 
thè value of n 2 by 2 n + 1 , a value that increases as n increases. 

n 2 is not in 0(n): « 2 is not in O(n). 
n — 2 is not in ©(Fibonacci(n + 1)): n — 2 is not in Cì(n). 

Fibonacci(n) is not in 0(n): Fibonacci(n + 1) is not in (Din — 2). 


Properties of O, O, and 0. Because O, O, and 0 are concerned with thè asymp- 
totic properties of functions, that is, how they grow as inputs approach infinity, 
many functions that are different when thè actual output values matter gener- 
ate identical sets with thè O, O, and 0 functions. For example, we saw n — 7 is in 
0(n + 12) and n + 12 is in ©(n — 7). In fact, every function that is in @(n — 7) is 
also in 0(n + 12). 

More generally, if we could prove g is in ©(an + k) where a is a positive Constant 
and/c is any Constant, theny is also in0(n). Thus, thè set ©(an + k) is equivalent 
to thè set0(n). 

We prove ©(an + k) = 0(w) using thè dehnition of 0. To prove thè sets are 
equivalent, we need to show inclusion in both directions. 

0(m) C &(an + k): For any function g, if g is in 0(n) then g is in &(an + k). 
Since g is in 0(«) there exist positive constants ci, C 2 , and no such that c\n > 
g(n) > C 2 n. To showy is also in Q(an + k) we find d\, d 2 , and mo such that 
di(an + k) > g(n ) > d 2 (an + k) for all n > m 0 . Simplifying thè inequalities, 
we need {ad\)n + kd\ > g(n) > (ad 2 )n + kdj. Ignoring thè constants for 
now, we can pick d\ — c -L an d d 2 — 2 since g is in 0 (n), we know 

(a—)n > g(n ) > (a—)n 
a a 

is satisfied. As for thè constants, as n increases they become insignihcant. 
Adding one to d\ and c ?2 adds an to thè first term and k to thè second term. 
Hence, as n grows, an becomes greater than k. 

0( an + k) C 0(k): For any function g, if g is in ©(an + k ) then g is in 0(n). 
Since g is in ©(an + k) there exist positive constants c\, C 2 , and n 0 such 
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that c\(an +k) > g(n) > c 2 (an + k). Simplifying thè inequalities, we have 
{ac\)n + kc\ > g(n ) > (ac 2 )n + kc 2 or, for some different positive constants 
b\ — ac\ and b 2 — ac 2 and constants k\ — kc\ and k 2 — kc 2 , b\n + /ci > 
g(n) > + k 2 - To showy is also in 0(«), we find di, d 2 , and nto such that 

di n > g(n) > d 2 n for all n > m 0 . If it were not for thè constants, we ai- 
ready have this with di — hi and d 2 — b 2 . As before, thè constants become 
inconsequential as n increases. 

This property also holds for thè O and fi operators since our proof for 0 also 
proved thè property for thè O and 0 inequalities. 

This result can be generalized to any polynomial. The set 0(«o + n i n + CI 211 2 + 
... + a k n k ) is equivalent to G(n k ). Because we are concerned with thè asymptotic 
growth, only thè highest power term of thè polynomial matters once ri gets big 
enough. 

Exercise 7.6. Repeat Exercise 7.2 using 0 instead of O. 

Exercise 7.7. Show that G[n 2 — n ) is equivalent to Gin 1 ), 

Exercise 7.8. [*] Is G(n 2 ) equivalent to 0(n 21 )? Either prove they are identical, 
or prove they are different. 

Exercise 7.9. [*] Is 0(2") equivalent to 0(3")? Either prove they are identical, or 
prove they are different. 

7.3 Analyzing Procedures 

By considering thè asymptotic growth of functions, rather than their actual out- 
puts, thè O, 0, and 0 operators allow us to hide constants and factors that 
change depending on thè speed of our processor, how data is arranged in mem- 
ory, and thè specifics of how our interpreter is implemented. Instead, we can 
consider thè essential properties of how thè running time of thè procedures in- 
creases with thè size of thè input. 

This section explains how to measure input sizes and running times. To under- 
stand thè growth rate of a procedure’s running time, we need a function that 
maps thè size of thè inputs to thè procedure to thè amount of time it takes to 
evaluate thè application. First we consider how to measure thè input size; then, 
we consider how to measure thè running time. In Section 7.3.3 we consider 
which input of a given size should be used to reason about thè cost of applying 
a procedure. Section 7.4 provides examples of procedures with different growth 
rates. The growth rate of a procedure’s running time gives us an understanding 
of how thè running time increases as thè size of thè input increases. 

7.3.1 Input Size 

Procedure inputs may be many different types: Numbers, Lists of Numbers, 
Lists of Lists, Procedures, etc. Our goal is to characterize thè input size with a 
single number that does not depend on thè types of thè input. 

We use thè Turing machine to model a computer, so thè way to measure thè size 
of thè input is thè number of characters needed to write thè input on thè tape. 
The characters can be from any fixed-size alphabet, such as thè ten decimai dig- 
its, or thè letters of thè alphabet. The number of different symbols in thè tape 
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alphabet does not matter for our analysis since we are concerned with orders of 
growth not absolute values. Within thè O, O, and 0 operators, a Constant fac- 
tor does not matter (e.g., 0(«) = 0(17n + 523)). This means is doesn’t matter 
whether we use an alphabet with two symbols or an alphabet with 256 symbols. 
With two symbols thè input may be 8 times as long as it is with a 256-symbol al- 
phabet, but thè Constant factor does not matter inside thè asymptotic operator. 

Thus, we measure thè size of thè input as thè number of symbols required to 
write thè number on a Turing Machine input tape. To figure out thè input size 
of a given type, we need to think about how many symbols it would require to 
write down inputs of that type. 

Booleans. There are only two Boolean values: true and false. Hence, thè length 
of a Boolean input is fìxed. 

Numbers. Using thè decimai number System (that is, 10 tape symbols), we can 
write a number of magnitude n using log 10 n digits. Using thè binary number 
System (that is, 2 tape symbols), we can write it using log 2 n bits. Within thè 
asymptotic operators, thè base of thè logarithm does not matter (as long as it is 
a Constant) since it changes thè result by a Constant factor. We can see this from 
thè argument above — changing thè number of symbols in thè input alphabet 
changes thè input length by a Constant factor which has no impact within thè 
asymptotic operators. 

Lists. If thè input is a List, thè size of thè input is related to thè number of 
elements in thè list. If each element is a Constant size (for example, a list of 
numbers where each number is between 0 and 100), thè size of thè input list is 
some Constant multiple of thè number of elements in thè list. Hence, thè size of 
an input that is a list of « elements iscn for some constante. Since0(cn) = &(n), 
thè size of a List input is 0(«) where n is thè number of elements in thè List. If 
List elements can vary in size, then we need to account for that in thè input size. 
For example, suppose thè input is a List of Lists, where there are n elements in 
each inner List, and there are n List elements in thè main List. Then, there are n 2 
total elements and thè input size is in @(n 2 ). 

7.3.2 Running Time 

We want a measure of thè running time of a procedure that satisfìes two proper- 
ties: (1) it should be robust to ephemeral properties of a particular execution or 
computer, and (2) it should provide insights into how long it takes evaluate thè 
procedure on a wide range of inputs. 

To estimate thè running time of an evaluation, we use thè number of steps re- 
quired to perform thè evaluation. The actual number of steps depends on thè 
details of how much work can be done on each step. For any particular proces- 
sor, both thè time it takes to perform a step and thè amount of work that can be 
done in one step varies. When we analyze procedures, however, we usually don’t 
want to deal with these details. Instead, what we care about is how thè running 
time changes as thè input size increases. This means we can count anything we 
want as a “step” as long as each step is thè approximately same size and thè time 
a step requires does not depend on thè size of thè input. 

The clearest and simplest definition of a step is to use one Turing Machine step. 
We have a precise definition of exactly what a Turing Machine can do in one step: 


138 


7.3. Analyzing Procedures 


Time makes more 
converts than 
reason. 
Thomas Paine 


worst case 


it can read thè Symbol in thè current square, write a Symbol into that square, 
transition its internai state number, and move one square to thè left or right. 
Counting Turing Machine steps is very precise, but difficult because we do not 
usually start with a Turing Machine description of a procedure and creating one 
is tedious. 

Instead, we usually reason directly from a Scheme procedure (or any precise de- 
scription of a procedure) using larger steps. As long as we can claim that what- 
ever we consider a step could be simulated using a Constant number of steps 
on a Turing Machine, our larger steps will produce thè same answer within thè 
asymptotic operators. One possibility is to count thè number of times an evalua- 
tion rule is used in an evaluation of an application of thè procedure. The amount 
of work in each evaluation rule may vary slightly (for example, thè evaluation 
rule for an if expression seems more complex than thè rule for a primitive) but 
does not depend on thè input size. 

Hence, it is reasonable to assume all thè evaluation rules to take Constant time. 
This does not include any additional evaluation rules that are needed to apply 
one rule. For example, thè evaluation rule for application expressions includes 
evaluating every subexpression. Evaluating an application constitutes one work 
unit for thè application rule itself, plus all thè work required to evaluate thè 
subexpressions. In cases where thè bigger steps are unclear, we can always re- 
turn to our precise definition of a step as one step of a Turing Machine. 

7.3.3 Worst Case Input 

A procedure may have different running times for inputs of thè same size. 

For example, consider this procedure that takes a List as input and outputs thè 
first positive number in thè list: 

(define ( list-fìrst-pos p) 

(if ( nuli ? p) ( error "No positive element found") 

(if (> ( car p) 0) ( car p) ( list-fìrst-pos ( cdr p))))) 

If thè first element in thè input list is positive, evaluating thè application of list- 
fìrst-pos requires very little work. It is not necessary to consider any other ele- 
ments in thè list if thè first element is positive. On thè other hand, if none of thè 
elements are positive, thè procedure needs to test each element in thè list until 
it reaches thè end of thè list (where thè base case reports an error). 

In our analyses we usually consider thè worst case input. For a given size, thè 
worst case input is thè input for which evaluating thè procedure takes thè most 
work. By focusing on thè worst case input, we know thè maximum running time 
for thè procedure. Without knowing something about thè possible inputs to 
thè procedure, it is safest to be pessimistic about thè input and not assume any 
properties that are not known (such as that thè first number in thè list is positive 
for thè flrst-pos example). 

In some cases, we also consider thè average case input. Since most procedures 
can take infinitely many inputs, this requires understanding thè distribution of 
possible inputs to determine an “average” input. This is often necessary when 
we are analyzing thè running time of a procedure that uses another helper pro- 
cedure. If we use thè worst-case running time for thè helper procedure, we will 
grossly overestimate thè running time of thè main procedure. Instead, since 
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we know how thè main procedure uses thè helper procedure, we can more pre- 
cisely estimate thè actual running time by considering thè actual inputs. We see 
an example of this in thè analysis of how thè + procedure is used by list-length 
in Section 7.4.2. 

7.4 Growth Rates 

Since our goal is to understand how thè running time of an application of a pro- 
cedure is related to thè size of thè input, we want to devise a function that takes 
as input a number that represents thè size of thè input and outputs thè maxi- 
mum number of steps required to complete thè evaluation on an input of that 
size. Symbolically, we can think of this function as: 

Max-Stepsp roc : Number —? Number 

where Proc is thè name of thè procedure we are analyzing. Because thè output 
represents thè maximum number of steps required, we need to consider thè 
worst-case input of thè given size. 

Because of all thè issues with counting steps exactly, and thè uncertainty about 
how much work can be done in one step on a particular machine, we cannot 
usually determine thè exact function for Max-Stepsp mc . Instead, we charac- 
terize thè running time of a procedure with a set of functions denoted by an 
asymptotic operator. Inside thè O, O, and 0 operators, thè actual time needed 
for each step does not matter since thè Constant factors are hidden by thè oper- 
ator; what matters is how thè number of steps required grows as thè size of thè 
input grows. 

Hence, we will characterize thè running time of a procedure using a set of func- 
tions produced by one of thè asymptotic operators. The 0 operator provides thè 
most information. Since 0(/) is thè intersection of O(f) (no faster than) and 
Q(/) (no slower than), knowing that thè running time of a procedure is in 0(/) 
for some function / provides much more information than just knowing it is in 
O(f) or just knowing that it is in Q(/). Hence, our goal is to characterize thè 
running time of a procedure using thè set of functions defined by 0(/) of some 
function /. 

The rest of this section provides examples of procedures with different growth 
rates, from slowest (no growth) through increasingly rapid growth rates. The 
growth classes described are important classes that are commonly encountered 
when analyzing procedures, but these are only examples of growth classes. Be- 
tween each pair of classes described here, there are an unlimited number of dif- 
ferent growth classes. 

7.4.1 No Growth: Constant Time 

If thè running time of a procedure does not increase when thè size of thè input 
increases, thè procedure must be able to produce its output by looking at only a 
Constant number of symbols in thè input. Procedures whose running time does 
not increase with thè size of thè input are known as Constant time procedures. 
Their running time is in 0(1) — it does not grow at all. By convention, we use 
0(1) instead of 0(1) to describe Constant time. Since there is no way to grow 
slower than not growing at all, 0(1) and 0(1) are equivalent. 


Constant time 
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We cannot do much in Constant time, since we cannot even examine thè whole 
input. A Constant time procedure must be able to produce its output by exam- 
ining only a fìxed-size part of thè input. Recali that thè input size measures thè 
number of squares needed to represent thè input. No matter how long thè input 
is, a Constant time procedure can look at no more than some fixed number of 
squares on thè tape, so cannot even read thè whole input. 

An example of a Constant time procedure is thè built-in procedure car. When 
car is applied to a non-empty list, it evaluates to thè first element of that list. 
No matter how long thè input list is, all thè car procedure needs to do is extract 
thè first component of thè list. So, thè running time of car is in 0(1). 4 Other 
built-in procedures that involve lists and pairs that have running times in 0(1) 
include cons, cdr, nuli?, and pair?. None of these procedures need to examine 
more than thè first pair of thè list. 

7.4.2 Linear Growth 

When thè running time of a procedure increases by a Constant amount when thè 
linearly size of thè input grows by one, thè running time of thè procedure grows linearly 
with thè input size. If thè input size is n, thè running time is in 0(n). If a proce- 
dure has running time in 0(n), doubling thè size of thè input will approximately 
doublé thè execution time. 

An example of a procedure that has linear growth is thè elementary school ad- 
dition algorithm from Section 6.2.3. To add two d-digit numbers, we need to 
perform a Constant amount ofwork for each digit. The number of steps required 
grows linearly with thè size of thè numbers (recali from Section 7.3. 1 that thè size 
of a number is thè number of input symbols needed to represent thè number). 

Many procedures that take a List as input have linear time growth. A procedure 
that does something that takes Constant time with every element in thè input 
List, has running time that grows linearly with thè size of thè input since adding 
one element to thè list increases thè number of steps by a Constant amount. 
Next, we analyze three list procedures, all of which have running times that scale 
linearly with thè size of their input. 


Example 7.4: Append 


Consider thè list-append procedure (from Example 5.6): 

(define ( list-append p q) 

(if ( nuli ? p) q ( cons ( car p) ( list-append ( cdr p) q)))) 

Since list-append takes two inputs, we need to be careful about how we refer 
to thè input size. We use n f , to represent thè number of elements in thè first 
input, and n q to represent thè number of elements in thè second input. So, our 
goal is to define a function Max-Stepsp st _ a ^ enc ^{n p ,n q ) that captures how thè 
maximum number of steps required to evaluate an application of list-append 
scales with thè size of its input. 


4 Since we are speculating based on what car does, not examining how car a particular Scheme 
interpreter actuaUy implements it, we cannot say definitively that its running time is in 0(1). It 
would be rather shocking, however, for an implementation to implement car in a way such that 
its running time that is not in 0(1). The implementation of scar in Section 5.2.1 is Constant time: 
regardless of thè input size, evaluating an application of it involves evaluating a single application 
expression, and then evaluating an if expression. 
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To analyze thè running time of list-append, we examine its body which is an if 
expression. The predicate expression applies thè nuli? procedure with is Con- 
stant time since thè effort required to determine if a list is nuli does not depend 
on thè length of thè list. When thè predicate expression evaluates to true, thè 
alternate expression is just q, which can also be evaluated in Constant time. 

Next, we consider thè alternate expression. It includes a recursive application 
of list-append. Hence, thè running time of thè alternate expression is thè time 
required to evaluate thè recursive application plus thè time required to evaluate 
everything else in thè expression. The other expressions to evaluate are applica- 
tions of cons, car, and cdr, all of which is are Constant time procedures. 

So, we can defined thè total running time recursively as: 

Max-Steps p st _ a pp en( i (n p , n p ) = C + Max-Stepsp st _ a pp enc ^(n p — 1 ,riq) 

where C is some Constant that reflects thè time for all thè operations besides thè 
recursive cali. Note that thè value of does not matter, so we simplify this to: 

Max-Steps ij st _ a pp en( j(np) = C + Max-Stepsp st _ a pp en ^(n p — 1). 

This does not yet provide a useful characterization of thè running time of list- 
append though, since it is a circular definition. To make it a recursive defìnition, 
we need a base case. The base case for thè running time definition is thè same 
as thè base case for thè procedure: when thè input is nuli. For thè base case, thè 
running time is Constant: 

Max-Steps p st _ a pp en( ^(0) — Co 

where Co is some Constant. 

To better characterize thè running time of list-append, we want a closed form 
solution. For a given input n, Max-Steps(n) isC + C + C + C + ... + C + Co where 
there are n — 1 of thè C terms in thè sum. This simplifies to (n — 1)C + Co = 
nC — C + Co — nC + C 2 - We do not know what thè values of C and C 2 are, but 
within thè asymptotic notations thè Constant values do not matter. The impor- 
tant property is that thè running time scales linearly with thè value of its input. 
Thus, thè running time of list-append is in 0(n p ) where n p is thè number of 
elements in thè first input. 

Usually, we do not need to reason at quite this low a level. Instead, to analyze thè 
running time of a recursive procedure it is enough to determine thè amount of 
work involved in each recursive cali (excluding thè recursive application itself) 
and multiply this by thè number of recursive calls. For this example, there are n p 
recursive calls since each cali reduces thè length of thè p input by one until thè 
base case is reached. Each cali involves only constant-time procedures (other 
than thè recursive application), so thè amount of work involved in each cali is 
Constant. Hence, thè running time is in 0(n p ). Equivalently, thè running time 
for thè list-append procedure scales linearly with thè length of thè first input list. 


Example 7.5: Length 


Consider thè list-length procedure from Example 5.1: 

(define ( list-length p) (if ( nuli ? p) 0 (+ 1 ( list-length ( cdr p ) ) ) ) ) 
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This procedure makes one recursive application of list-length for each element 
in thè input p. If thè input has n elements, there will ben + 1 total applications 
of list-length to evaluate (one for each element, and one for thè nuli). So, thè 
total work is in 0(« ■ work for each recursive application). 

To determine thè running time, we need to determine how much work is in- 
volved in each application. Evaluating an application of list-length involves eval- 
uating its body, which is an if expression. To evaluate thè if expression, thè pred- 
icate expression, ( nuli ? p), must be evaluated first. This requires Constant time 
since thè nuli ? procedure has Constant running time (see Section 7.4.1). The 
consequent expression is thè primitive expression, 0, which can be evaluated in 
Constant time. The alternate expression, (+ 1 ( list-length ( cdr p))), includes thè 
recursive cali. There are n + 1 total applications of list-length to evaluate, thè 
total running time is n + 1 times thè work required for each application (other 
than thè recursive application itself). 

The remaining work is evaluating ( cdr p) and evaluating thè + application. The 
cdr procedure is Constant time. Analyzing thè running time of thè + procedure 
application is more complicated. 

Cost of Addition. Since + is a built-in procedure, we need to think about how 
it might be implemented. Following thè elementary school addition algorithm 
(from Section 6.2.3), we know we can add any two numbers by walking down 
thè digits. The work required for each digit is Constant; we just need to compute 
thè corresponding result and carry bits using a simple formula or lookup table. 
The number of digits to add is thè maximum number of digits in thè two input 
numbers. Thus, if there are b digits to add, thè total work is in &(b). In thè worst 
case, we need to look at all thè digits in both numbers. In generai, we cannot 
do asymptotically better than this, since adding two arbitrary numbers might 
require looking at all thè digits in both numbers. 

But, in thè list-length procedure thè + is used in a very limited way: one of thè 
inputs is always 1 . We might be able to add 1 to a number without looking at all 
thè digits in thè number. Recali thè addition algorithm: we start at thè rightmost 
(least signihcant) digit, add that digit, and continue with thè carry. If one of 
thè input numbers is 1 , then once thè carry is zero we know now of thè more 
signihcant digits will need to change. In thè worst case, adding one requires 
changing every digit in thè other input. For example, (+ 99999 1) is 100000. In 
thè best case (when thè last digit is below 9) , adding one requires only examining 
and changing one digit. 

Figuring out thè average case is more difhcult, but necessary to get a good esti- 
mate of thè running time of list-length. We assume thè numbers are represented 
in binary, so instead of decimai digits we are counting bits (this is both simpler, 
and closer to how numbers are actually represented in thè computer). Approx- 
imately half thè time, thè least signihcant bit is a 0, so we only need to examine 
one bit. When thè last bit is not a 0, we need to examine thè second least signih- 
cant bit (thè second bit from thè right) : if it is a 0 we are done; if it is a 1 , we need 
to continue. 

We always need to examine one bit, thè least signihcant bit. Half thè time we 
also need to examine thè second least signihcant bit. Of those times, half thè 
time we need to continue and examine thè next least signihcant bit. This con- 
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tinues through thè whole number. Thus, thè expected number of bits we need 
to examine is, 





where thè number of terms is thè number of bits in thè input number, b. Simpli- 
fying thè equation, we get: 


1 


1111 
2 + 4 + 8 + 16 


1 

¥ 


No matter how large b gets, this value is always less than 2. So, on average, thè 
number of bits to examine to add 1 is Constant: it does not depend on thè length 
of thè input number. 

This result generalizes to addition where one of thè inputs is any Constant value. 
Adding any Constant C to a number n is equivalent to adding one C times. Since 
adding one is a Constant time procedure, adding one C times can also be done 
in Constant time for any Constant C. 

Excluding thè recursive application, thè list-length application involves appli- 
cations of two Constant time procedures: cdr and adding one using +. Hence, 
thè total time needed to evaluate one application of list-length, excluding thè 
recursive application, is Constant. 

There are n + 1 total applications of list-length to evaluate total, so thè total run- 
ning time is c(n + 1) where c is thè amount of time needed for each application. 
The set &(c(n + 1)) is identicalto thè set ©(«), so thè running time for thè length 
procedure is in 0(«) where n is thè length of thè input list. 


Example 7.6: Accessing List Elements 


Consider thè list-get-element procedure from Example 5.3: 

(define ( list-get-element p n) 

(if(= ni) 

( car p ) 

( list-get-element ( cdr p) (- n 1)))) 

The procedure takes two inputs, a List and a Number selecting thè element of 
thè list to get. Since there are two inputs, we need to think carefully about thè 
input size. We can use variables to represent thè size of each input, for example 
Sp and s n for thè size of p and n respectively. In this case, however, only thè size 
of thè first input really matters. 

The procedure body is an if expression. The predicate uses thè built-in = pro- 
cedure to compare n to 1 . The worst case running time of thè — procedure is 
linear in thè size of thè input: it potentially needs to look at all bits in thè input 
numbers to determine if they are equal. Similarly to +, however, if one of thè 
inputs is a Constant, thè comparison can be done in Constant time. To compare 
a number of any size to 1 , it is enough to look at a few bits. If thè least significant 
bit of thè input number is not a 1 , we know thè result is false. If it is a 1 , we need 
to examine a few other bits of thè input number to determine if its value is dif- 
ferent from 1 (thè exact number of bits depends on thè details of how numbers 
are represented). So, thè = comparison can be done in Constant time. 
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If thè predicate is true, thè base case applies thè car procedure, which has Con- 
stant running time. The alternate expression involves thè recursive calls, as well 
as evaluating ( cdr p), which requires Constant time, and (— ni). The — proce- 
dure is similar to +: for arbitrary inputs, its worst case running time is linear 
in thè input size, but when one of thè inputs is a Constant thè running time is 
Constant. This follows from a similar argument to thè one we used for thè + 
procedure (Exercise 7.13 asks for a more detailed analysis of thè running time of 
subtraction) . So, thè work required for each recursive cali is Constant. 

The number of recursive calls is determined by thè value of n and thè number 
of elements in thè list p. In thè best case, when n is 1 , there are no recursive calls 
and thè running time is Constant since thè procedure only needs to examine 
thè first element. Each recursive cali reduces thè value passed in as n by 1 , so 
thè number of recursive calls scales linearly with n (thè actual number is n — 1 
since thè base case is when n equals 1 ). But, there is a limit on thè value of n for 
which this is true. If thè value passed in as n exceeds thè number of elements in 

р, thè procedure will produce an error when it attempts to evaluate ( cdr p ) for 
thè empty list. This happens after s p recursive calls, where s p is thè number of 
elements in p. Hence, thè running time of list- get- element does not grow with 
thè length of thè input passed as rr, after thè value of n exceeds thè number of 
elements in p it does not matter how much bigger it gets, thè running time does 
not continue to increase. 

Thus, thè worst case running time of list-get-element grows linearly with thè 
length of thè input list. Equivalently, thè running time of list-get-element is in 
0(s p ) where s p is thè number of elements in thè input list. 

Exercise 7.10. Explain why thè list-map procedure from Section 5.4.1 has run- 
ning time that is linear in thè size of its List input. Assume thè procedure input 
has Constant running time. 

Exercise 7.11. Consider thè list-sum procedure (from Example 5.2): 

(define ( list-sum p) (if ( nuli ? p) 0 (+ ( car p) ( list-sum ( cdr p) ) ) ) ) 

What assumptions are needed about thè elements in thè list for thè running time 
to be linear in thè number if elements in thè input list? 

Exercise 7.12. For thè decimai six-digit odometer (shown in thè picture on 
page 142), we measure thè amount of work to add one as thè total number of 
wheel digit tums required. For example, going from 000000 to 000001 requires 
one work unit, but going from 000099 to 0001 00 requires three work units. 

a. What are thè worst case inputs? 

b. What are thè best case inputs? 

с. [*] On average, how many work units are required for each mile? Assume 
over thè lifetime of thè odometer, thè car travels 1,000,000 miles. 

d. Lever voting machines were used by thè majority of American voters in thè 
1960s, although they are not widely used today. Most level machines used a 
three-digit odometer to tally votes. Explain why candidates ended up with 99 
votes on a machine far more often than 98 or 100 on these machines. 
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Exercise 7.13. [*] The list-get-element argued by comparison to +, that thè — 
procedure has Constant running time when one of thè inputs is a Constant. De- 
velop a more convincing argument why this is true by analyzing thè worst case 
and average case inputs for — . 

Exercise 7.14. *] Our analysis of thè work required to add one to a number ar- 
gued that it could be done in Constant time. Test experimentally if thè DrRacket 
+ procedure actually satisfies this property. Note that one + application is too 
quick to measure well using thè time procedure, so you will need to design a 
procedure that applies + many times without doing much other work. 

7.4.3 Quadratic Growth 

If thè running time of a procedure scales as thè square of thè size of thè input, 
thè procedure’s running time grows quadratically. Doubling thè size of thè in- 
put approximately quadruples thè running time. The running time is in 0(n 2 ) 
where n is thè size of thè input. 

A procedure that takes a list as input has running time that grows quadratically 
if it goes through all elements in thè list once for every element in thè list. For 
example, we can compare every element in a list of length n with every other 
element using n(n — 1) comparisons. This simplihes to n 2 — n, but 0(n 2 — n ) 
is equivalent to 0(« 2 ) since as n increases only thè highest power term matters 
(see Exercise 7.7). 


Example 7.7: Reverse 


Consider thè list-reverse procedure defined in Section 5.4.2: 

(define ( list-reverse p ) 

(if ( nuli ? p) nuli ( list-append ( list-reverse ( cdr p)) ( list ( car p) ) ) ) ) 

To determine thè running time of list-reverse , we need to know how many recur- 
sive calls there are and how much work is involved in each recursive cali. Each 
recursive application passes in ( cdr p) as thè input, so reduces thè length of thè 
input list by one. Hence, applying list-reverse to a input list with n elements in- 
volves n recursive calls. 

The work for each recursive application, excluding thè recursive cali itself, is ap- 
plying list-append. The first input to list-append is thè output of thè recursive 
cali. As we argued in Example 7.4, thè running time of list-append is in 0(n p ) 
where n p is thè number of elements in its first input. So, to determine thè run- 
ning time we need to know thè length of thè first input list to list-append. For thè 
first cali, ( cdr p) is thè parameter, with length n — 1; for thè second cali, there will 
ben — 2 elements; and so forth, until thè final cali where ( cdr p) has 0 elements. 
The total number of elements in all of these calls is: 

(n — 1) + (n — 2) + . . . + 1 + 0. 

The average number of elements in each cali is approximately |. Within thè 
asymptotic operators thè Constant factor of \ does not matter, so thè average 
running time for each recursive application is in 0(n). 

There are n recursive applications, so thè total running time of list-reverse is n 


quadratically 
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times thè average running time of each recursive application: 

n ■ 0(n) = 0(n 2 ). 


Thus, thè running time is quadratic in thè size of thè input list. 


Example 7.8: Multiplication 


Consider thè problem of multiplying two numbers. The elementary school long 
multiplication algorithm works by multiplying each digit in b by each digit in a, 
aligning thè intermediate results in thè right places, and summing thè results: 


a n ~\ ■ ■ ■ ci\ ao 

x K-\ ■ ■ ■ h b 0 



+ a n-ib n -i 


fln - 1^0 ' ' ' a 1^0 a 0^0 

ci n —\b\ ■ ■ ■ ci\b\ a ob\ 

Cl\b n —\ flobn—i 


rm-i r 2n -2 r 3 r 2 n r 0 


If both input numbers have n digits, there are n 2 digit multiplications, each of 
which can be done in Constant time. The intermediate results will be n rows, 
each containing n digits. So, thè total number of digits to add is n 2 : 1 digit in thè 
ones place, 2 digits in thè tens place, . . ., n digits in thè 10" _1 s place, . . ., 2 digits 
in thè 10 2,,_3 s place, and 1 digit in thè 10 2,I_2 s place. Each digit addition requires 
Constant work, so thè total work for all thè digit additions is in 0(n 2 ). Adding 
thè work for both thè digit multiplications and thè digit additions, thè total run- 
ning time for thè elementary school multiplication algorithm is quadratic in thè 
number of input digits, 0(n 2 ) where n is thè number if digits in thè inputs. 


This is not thè fastest known algorithm for multiplying two numbers, although it 
was thè best algorithm known until 1960. In 1960, Anatolii Karatsuba discovers 
a multiplication algorithm with running time in 0(n lo Sz 3 ). Sincelog 2 3 < 1.585 
this is an improvement over thè 0(« 2 ) elementary school algorithm. In 2007, 
Martin Ftirer discovered an even faster algorithm for multiplication. 5 It is not 
yet known if this is thè fastest possible multiplication algorithm, or if faster ones 
exist. 


Exercise 7.15. [★] Analyze thè running time of thè elementary school long divi- 
sion algorithm. 

Exercise 7.16. [*] Define a Scheme procedure that multiplies two multi-digit 
numbers (without using thè built-in * procedure except to multiply single-digit 
numbers). Strive for your procedure to have running time in 0(n) where n is thè 
total number of digits in thè input numbers. 

Exercise 7.17. [* * **] Devise an asymptotically faster generai multiplication 
algorithm than Fiirer’s, or prove that no faster algorithm exists. 


5 Martin Fiirer, Faster Integer Multiplication, ACM Symposium on Theory of Computing, 2007. 





Chapter7. Cost 


147 


7.4.4 Exponential Growth 

If thè running time of a procedure scales as a power of thè size of thè input, 
thè procedure’s running time grows exponentially. When thè size of thè input 
increases by one, thè running time is multiplied by some Constant factor. The 
growth rate of a function whose output is multiplied by w when thè input size, 
n, increases by one is w n . Exponential growth is very fast — it is not feasible to 
evaluate applications of an exponential time procedure on large inputs. 

For a surprisingly large number of interesting problems, thè best known algo- 
rithm has exponential running time. Examples of problems like this include 
finding thè best route between two locations on a map (thè problem mentioned 
at thè beginning of Chapter 4), thè pegboard puzzle (Explo ratio n 5.2, solving 
generalized versions of most other games such as Suduko and Minesweeper, 
and finding thè factors of a number. Whether or not it is possible to design 
faster algorithms that solve these problems is thè most important open prob- 
lem in computer Science. 


Example 7.9: Factoring 


A simple way to find a factor of a given input number is to exhaustively try all 
possible numbers below thè input number to find thè first one that divides thè 
number evenly. The find- factor procedure takes one number as input and out- 
puts thè lowest factor of that number (other than 1 ) : 

(define ( find-factor n) 

(define ( find-factor-helper v) 

(if (= ( modulo n v) 0) v ( find-factor-helper (+ 1 c)))) 

( find-factor-helper 2)) 

The find-factor-helper procedure takes two inputs, thè number to factor and thè 
current guess. Since all numbers are divisible by themselves, thè modulo test 
will eventually be trae for any positive input number, so thè maximum number 
of recursive calls is ri, thè magnitude of thè input to find-factor. The magnitude 
of n is exponential in its size, so thè number of recursive calls is in 0(2 b ) where 
b is thè number of bits in thè input. This means even if thè amount of work re- 
quired for each recursive cali were Constant, thè running time of thè find-factor 
procedure is stili exponential in thè size of its input. 

The actual work for each recursive cali is not Constant, though, since it involves 
an application of modulo. The modulo built-in procedure takes two inputs and 
outputs thè remainder when thè first input is divided by thè second input. Hence, 
it output is 0 if n is divisible by v. Computing a remainder, in thè worst case, at 
least involves examining every bit in thè input number, so scales at least linearly 
in thè size of its input 6 . This means thè running time of find-factor is in Cì(2 b ): 
it grows at least as fast as 2 b . 

There are lots of ways we could produce a faster procedure for finding factors: 
stopping once thè square root of thè input number is reached since we know 
there is no need to check thè rest of thè numbers, skipping even numbers after 2 
since if a number is divisible by any even number it is also divisible by 2, or using 
advanced sieve methods. This techniques can improve thè running time by Con- 
stant factors, but there is no known factoring algorithm that runs in faster than 

6 In fact, it computing thè remainder requires performing division, which is quadratic in thè size 
of thè input. 
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exponential time. The security of thè widely used RSA encryption algorithm de- 
pends on factoring being hard. If someone frnds a fast factoring algorithm it 
would put thè codes used to secure Internet commerce at risk. 7 


Example 7.10: Power Set 


power set The power set of a set S is thè set of all subsets of S. For example, thè power set 
of {1,2,3} is {{}, {1}, {2}, {3}, {1,2}, {1,3}, {2,3}, {1,2,3}}. 

The number of elements in thè power set of S is 2l s l (where |S| is thè number of 
elements in thè set S). 

Here is a procedure that takes a list as input, and produces as output thè power 
set of thè elements of thè list: 

(define ( list-powerset s ) 

(if ( nuli ? s) ( list nuli ) 

( list-append ( list-map (lambda (f) ( cons ( car s) t)) 

( list-powerset ( cdr s))) 

( list-powerset ( cdr 5 ))))) 

The list-powerset procedure produces a List of Lists. Hence, for thè base case, 
instead of just producing nuli, it produces a list containing a single element, 
nuli In thè recursive case, we can produce thè power set by appending thè list 
of all thè subsets that include thè first element, with thè list of all thè subsets that 
do not include thè first element. For example, thè powerset of {1,2,3} is found 
by finding thè powerset of {2,3}, which is {{}, {2}, {3}, {2, 3}}, and taking thè 
union of that set with thè set of all elements in that set unioned with {1}. 

An application of list-powerset involves applying list-append, and two recursive 
applications of ( list-powerset ( cdr s]). Increasing thè size of thè input list by one, 
doubles thè total number of applications of list-powerset since we need to eval- 
uate ( list-powerset ( cdr s)) twice. The number of applications of list-powerset is 
2" where n is thè length of thè input list. 8 

The body of list-powerset is an if expression. The predicate applies thè constant- 
time procedure, nuli?. The consequent expression, [list nuli ) is also Constant 
time. The alternate expression is an application of list-append. From Exam- 
ple 7.4, we know thè running time of list-append is 0(n p ) where n p is thè num- 
ber of elements in its first input. The first input is thè result of applying list-map 
to a procedure and thè List produced by ( list-powerset ( cdr s)). The length of 
thè list output by list-map is thè same as thè length of its input, so we need to 
determine thè length of ( list-powerset ( cdr s)). 

We use n s to represent thè number of elements in s. The length of thè input list 
to map is thè number of elements in thè power set of a size n s - 1 set: 2” s_1 . But, 
for each application, thè value of n s is different. Since we are trying to determine 
thè total running time, we can do this by thinking about thè total length of all thè 
input lists to list-map over all of thè list-powerset. In thè input is a list of length 
n, thè total list length is 2 n_1 + 2" -2 + ... + 2 1 + 2°, which is equal to 2" — 1. So, 


7 The movie Sneakers is a fictional account of what would happen if someone frnds a faster than 
exponential time factoring algorithm. 

8 Observant readers will note that it is not really necessary to perform this evaluation twice since 
we could do it once and reuse thè result. Even with this change, though, thè running time would stili 
be in 0(2"). 
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thè running tinte for all thè list-map applications is in 0(2"). 

The analysis of thè list-append applications is similar. The length of thè hrst 
input to list-append is thè length of thè result of thè list-powerset application, 
so thè total length of all thè inputs to append is 2". 

Other than thè applications of list-map and list-append, thè rest of each list- 
powerset application requires Constant tinte. So, thè running tinte required for 
2" applications is in 0(2"). The total running tinte for list-powerset is thè sum 
of thè running times for thè list-powerset applications, in 0(2"); thè list-map 
applications, in 0(2"); and thè list-append applications, in 0(2"). Hence, thè 
total running tinte is in 0(2"). 

In this case, we know there can be no faster than exponential procedure that 
solves thè sante problem, since thè size of thè output is exponential in thè size 
of thè input. Since thè most work a Turing Machine can do in one step is write 
one square, thè size of thè output provides a lower bound on thè running tinte 
of thè Turing Machine. The size of thè powerset is 2" where n is thè size of thè 
input set. Hence, thè fastest possible procedure for this problem has at least 
exponential running time. 


7.4.5 Faster than Exponential Growth 

We have already seen an example of a procedure that grows faster than expo- 
nentially in thè size of thè input: thè fibo procedure at thè beginning of this 
chapter! Evaluating an application of fibo involves &(<p n ) recursive applications 
where n is thè magnitude of thè input parameter. The size of a numeric input is 
thè number of bits needed to express it, so thè value n can be as high as 2 b — 1 
where b is thè number of bits. Hence, thè running time of thè fibo procedure is 
in ©(</>-’ ) where b is thè size of thè input. This is why we are stili waiting for ( fibo 
60) to finish evaluating. 

7.4.6 Non-terminating Procedures 

All of thè procedures so far in thè section are algorithms: they may be slow, but 
they are guaranteed to eventually finish if one can wait long enough. Some pro- 
cedures never terminate. For example, 

(define ( run-forever ) ( run-forever )) 

defines a procedure that never finishes. Its body calls itself, never making any 
progress toward a base case. The running time of this procedure is effectively 
infinite since it never finishes. 

7.5 Summary 

Because thè speed of computers varies and thè exact time required for a particu- 
lar application depends on many details, thè most important property to under- 
stand is how thè work required scales with thè size of thè input. The asymptotic 
operators provide a convenient way of understanding thè cost involved in eval- 
uating a procedure applications. 

Procedures that can produce an output only touching a fixed amount have Con- 
stant running times. Procedures whose running times increase by a fixed amount 
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when thè input size increases by one have linear (in 0(n)) running times. Proce- 
dures whose running tinte quadruples when thè input size doubles have quadratic 
(in 0(n 2 )) running times. Procedures whose running time doubles when thè in- 
put size increases by one have exponential (in 0(2" )) running times. Procedures 
with exponential running time can only be evaluated for small inputs. 

Asymptotic analysis, however, must be interpreted cautiously. For large enough 
inputs, a procedure with running time in 0(«) is always faster than a procedure 
with running time in 0(n 2 ). But, for an input of a particular size, thè 0(n 2 ) 
procedure may be faster. Without knowing thè constants that are hidden by thè 
asymptotic operators, there is no way to accurately predict thè actual running 
time on a given input. 


Exercise 7.18. Analyze thè asymptotic running time of thè list-sum procedure 
(from Example 5.2): 

(define ( list-sum p) 

(if ( nuli ? p) 

0 

(+ ( car p) ( list-sum ( cdr p))))) 

You may assume all of thè elements in thè list have values below some Constant 
(but explain why this assumption is useful in your analysis). 

Exercise 7.19. Analyze thè asymptotic running time of thè factorial procedure 
(from Example 4.1): 

(define ( factorial ri) (if (= n 0) 1 (* n ( factorial (— n 1))))) 

Be careful to describe thè running time in terms of thè size (not thè magnitude) 
of thè input. 

Exercise 7.20. Consider thè intsto problem (from Example 5.8). 

a. [*] Analyze thè asymptotic running time of this intsto procedure: 

(define {re virus io ri) 

(if (= n 0) 
nuli 

( cons n ( revintsto (- n 1))))) 

(define ( intsto ri) ( list-reverse ( revintsto ri))) 

b. [*] Analyze thè asymptotic running time of this instto procedure: 

(define ( intsto ri) 

(if (= n 0) nuli ( list-append ( intsto (— ni)) (list n)))) 

c. Which version is better? 

d. [**] Is there an asymptotically faster intsto procedure? 
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Exercise 7.21. Analyze thè running time of thè board-replace-peg procedure 
(from Exploration 5.2): 

(define ( row- replace-peg pegs col vai) 

(if (= col 1) ( cons vai ( cdr pegs)) 

(■ cons ( car pegs) ( row-replace-peg ( cdr pegs) (- col 1) vai)))) 

(define ( board-replace-peg board row col vai) 

(if (= row 1) ( cons ( row-replace-peg ( car board) col vai) ( cdr board)) 

(i cons ( car board) ( board-replace-peg ( cdr board) (— row 1) col vai)))) 

Exercise 7.22. Analyze thè running time of thè deep-list-flatten procedure from 
Section 5.5: 

(define ( deep-list-flatten p) 

(if ( nuli ? p) nuli 

( list-append (if [lisi? ( car p)) 

C deep-list-flatten ( car p)) 
llist ( car p))) 

(i deep-list-flatten ( cdr p))))) 

Exercise 7.23. [*] Find and correct at least one error in thè Orders of Growth 
section of thè Wikipedia page on Analysis of Algorithms (http://en.wikipedia.org/ 
wiki/AnalysiS-Of_algorithms). This is rated as [*] now (July 2011), since thè cur- 
rent entry contains many fairly obvious errors. Hopefully it will soon become a 
[* * *] challenge, and perhaps, eventually will become impossible! 


152 


7.5. Summary 


online library 


8 

Sorting and Searching 


Ifyou keep proving stuffthat others have dotte, getting confidence, increasing thè 
complexities ofyour Solutions — fot thefun ofit — then otte dayyou’ll turn 
around and discover that nobody actually did that one! 
And that's thè way to become a computer sdentisi 
Richard Feynman, Lectures on Computation 

This chapter presents two extended examples that use thè programming tech- 
niques from Chapters 2-5 and analysis ideas from Chapters 6-7 to solve some 
interesting and important problems: sorting and searching. These examples in- 
volve some quite challenging problems and incorporate many of thè ideas we 
have seen up to this point in thè hook. Once you understand them, you are well 
on your way to thinking like a computer scientisti 

8.1 Sorting 

The sorting problem takes two inputs: a list of elements and a comparison pro- 
cedure. It outputs a list containing same elements as thè input list ordered ac- 
cording to thè comparison procedure. For example, if we sort a list of numbers 
using < as thè comparison procedure, thè output is thè list of numbers sorted 
in order from least to greatest. 

Sorting is one of thè most widely studied problems in computing, and many 
different sorting algorithms have been proposed. Try to develop a sorting pro- 
cedure yourself before continuing further. It may be illuminating to try sorting 
some items by hand an think carefully about how you do it and how much worlc 
it is. For example, take a shuffled deck of cards and arrange them in sorted or- 
der by ranks. Or, try arranging all thè students in your class in order by birthday. 
Next, we present and analyze three different sorting procedures. 

8.1.1 Best- First Sort 

A simple sorting strategy is to hnd thè best element in thè list and put that at 
thè front. The best element is an element for which thè comparison procedure 
evaluates to true when applied to that element and every other element. For 
example, if thè comparison function is <, thè best element is thè lowest number 
in thè list. This element belongs at thè front of thè output list. 

The notion of thè best element in thè list for a given comparison function only 
makes sense if thè comparison function is transitive. This means it has thè prop- 
erty that for any inputs a, b, and c, if [cf a b) and ( cf b c) are both true, thè result 
of [cf a c ) must be true. The < function is transitive: a < b and b < c implies 
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a < c for all numbers a, b, and c. If thè comparison function does not have this 
property, there may be no way to arrange thè elements in a single sorted list. All 
of our sorting procedures require that thè procedure passed as thè comparison 
function is transitive. 

Once we can find thè best element in a given list, we can sort thè whole list by 
repeatedly finding thè best element of thè remaining elements until no more 
elements remain. To define our best-first sorting procedure, we fìrst define a 
procedure for finding thè best element in thè list, and then defìne a procedure 
for removing an element from a list. 

Finding thè Best. The best element in thè list is either thè fìrst element, or thè 
best element from thè rest of thè list. Hence, we defìne list-find-best recursively. 
An empty list has no best element, so thè base case is for a list that has one 
element. When thè input list has only one element, that element is thè best 
element. If thè list has more than one element, thè best element is thè better of 
thè fìrst element in thè list and thè best element of thè rest of thè list. 

To pick thè better element from two elements, we defìne thè pick-better proce- 
dure that takes three inputs: a comparison function and two values. 

(define ( pick-better cf pi p2 ) (if (c/ pi p2) pi p2)) 

Assuming thè procedure passed as cf has Constant running time, thè running 
time of pick-better is Constant. For most of our examples, we use thè < proce- 
dure as thè comparison function. For arbitrary inputs, thè running time of < is 
not Constant since in thè worst case performing thè comparison requires exam- 
ining every digit in thè input numbers. But, if thè maximum value of a number 
in thè input list is limited, then we can consider < a Constant time procedure 
since all of thè inputs passed to it are below some fìxed size. 

We use pick-better to defìne list-find-best: 

(define ( list-find-best cf p) 

(if ( nuli ? ( cdr p)) ( car p) 

( pick-better cf ( car p) ( list-find-best cf ( cdr p))))) 

We use n to represent thè number of elements in thè input list p. An applica- 
tion of list-find-best involves n — 1 recursive applications since each one passes 
in ( cdr p) as thè new p operand and thè base case stops when thè list has one 
element left. The running time for each application (excluding thè recursive 
application) is Constant since it involves only applications of thè Constant time 
procedures nuli?, cdr, and pick-better. So, thè total running time for list-find- 
best is in 0(n); it scales linearly with thè length of thè input list. 

Deleting an Element. To implement best fìrst sorting, we need to produce a 
list that contains all thè elements of thè originai list except for thè best element, 
which will be placed at thè front of thè output list. We defìne a procedure, list- 
delete, that takes as inputs a List and a Value, and produces a List that contains 
all thè elements of thè input list in thè originai order except for thè fìrst element 
that is equal to thè input value. 
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(define ( list-delete p el ) 

(if ( nuli ? p) nuli 

(if ( equal ? ( car p) el) ( cdr p) ; found match, skip this element 
( cons ( car p) ( list-delete ( cdr p) el))))) 

We use thè equal? procedure to check if thè element matches instead of = so 
thè list-delete procedure works on elements that are not just Numbers. The 
equal? procedure behaves identically to — when both inputs are Numbers, but 
also works sensibly on many other datatypes including Booleans, Characters, 
Pairs, Lists, and Strings. Since we assume thè sizes of thè inputs to equal? are 
bounded, we can consider equal? to be a Constant time procedure (even though 
it would not be Constant time on arbitrary inputs) . 

The worst case running time for list-delete occurs when no element in thè list 
matches thè value of el (in thè best case, thè first element matches and thè run- 
ning time does not depend on thè length of thè input list at all). We use n to 
represent thè number of elements in thè input list. There can be up to n re- 
cursive applications of list-delete. Each application has Constant running time 
since all of thè procedures applied (except thè recursive cali) have Constant run- 
ning times. Hence, thè total running time for list-delete is in 0(«) where n is thè 
length of thè input list. 

Best-First Sorting. We define list-sort-best-first using list-flnd-best and list- 
delete: 

(define ( list-sort-best-first cf p) 

(if ( nuli ? p) nuli 

( cons ( list-flnd-best cf p) 

( list-sort-best-first cf ( list-delete p ( list-flnd-best cf /?)))))) 

The running time of thè list-sort-best-first procedure grows quadratically with 
thè length of thè input list. We use n to represent thè number of elements in thè 
input list. There are n recursive applications since each application of list-delete 
produces an output list that is one element shorter than its input list. In addition 
to thè Constant time procedures ( nuli ? and cons), thè body of list-sort-best-first 
involves two applications of list-flnd-best on thè input list, and one application 
of list-delete on thè input list. 

Each of these applications has running time in 0(m) where m is thè length of 
thè input list to list-flnd-best and list-delete (we use m here to avoid confusion 
with n, thè length of thè first list passed into list-sort-best-first) . In thè first appli- 
cation, this input list will be a list of length n, but in later applications it will be 
involve lists of decreasing length: n — 1, n — 2, ■ ■ -, 1. Hence, thè average length 
of thè input lists to list-flnd-best and list-delete is approximately | . Thus, thè av- 
erage running time for each of these applications is in 0 ( |) , which is equivalent 
to &{n). 

There are three applications (two of list-flnd-best and one of list-delete) for each 
application of list-sort-best-first, so thè total running time for each application 
is in 0(3 n), which is equivalent to 0(n). 

There are n recursive applications, each with average running time in &(n), so 
thè running time for list-sort-best-first is in ®(n 2 ). This means doubling thè 
length of thè input list quadruples thè expected running time, so we predict that 
sorting a list of 2000 elements to take approximately four times as long as sorting 
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a list of 1000 elements. 

Let expression. Each application of thè list-sort-best-first procedure involves 
two evaluations of ( list-fìnd-best cf p), a procedure with running time in 0(n) 
where n is thè length of thè input list. 

The result of both evaluations is thè same, so there is no need to evaluate this 
expression twice. We could just evaluate ( list-find-best cf p) once and reuse thè 
result. One way to do this is to introduce a new procedure using a lambda ex- 
pression and pass in thè result of ( list-find-best cf p) as a parameter to this pro- 
cedure so it can be used twice: 

(define ( list-sort-best-flrst-nodup cf p) 

(if ( nuli ? p) nuli 
((lambda (best) 

( cons best ( list-sort-best-flrst-nodup cf ( list-delete p best )))) 

( list-fìnd-best cf p)))) 

This procedure avoids thè duplicate evaluation of ( list-find-best cf p), but is 
quite awkward to read and understand. 

Scheme provides thè let expression special form to avoid this type of duplicate 
work more elegantly. The grammar for thè let expression is: 


Expression 

:=> LetExpression 

LetExpression 

:=^> (let ( Bindings ) Expression ) 

Bindings 

:=>- Binding Bindings 

Bindings 

:=>- e 

Binding 

:=>• (Name Expression) 


The evaluation rule for thè let expression is: 

Evaluation Rule 6: Let expression. To evaluate a let expression, eval- 
uate each binding in order. To evaluate each binding, evaluate thè 
binding expression and bind thè name to thè value of that expres- 
sion. Then, thè value of thè let expression is thè value of thè body 
expression evaluated with thè names in thè expression that match 
binding names substituted with their bound values. 


A let expression can be transformed into an equivalent application expression. 
The let expression 

(let ((Name i Expression ^ ) (Name 2 Expression 2 ) 

■ ■ ■ (Name k Expression k )) 

Expression body ) 

is equivalent to thè application expression: 

((lambda (Name 1 Name 2 . . . Namef) Expression bod y) 

Expression 1 Expression 2 . . . Expression k ) 

The advantage of thè let expression syntax is it puts thè expressions next to thè 
names to which they are bound. Using a let expression, we define list- sor t-best- 
fìrst-let to avoid thè duplicate evaluations: 
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(define ( l is t- sort- b es L-fìrsL- lei cf p) 

(if ( nuli ? p) nuli 

(let {{best ( list-find-best cf p))) 

( cons best ( list-sort-best-first-let cf ( list-delete p best)))))) 

This runs faster than list-sort-best-first since it avoids thè duplicate evaluations, 
but thè asymptotic asymptotic running time is stili in 0(n 2 ): there are n recur- 
sive applications of list-sort-best-first-let and each application involves linear 
time applications of list-find-best and list-delete. Using thè let expression im- 
proves thè actual running time by avoiding thè duplicate work, but does not im- 
pact thè asymptotic growth rate since thè duplicate work is hidden in thè Con- 
stant faeton 


Exercise 8.1. What is thè best case input for list-sort-best-fìrstl What is its 
asymptotic running time on thè best case input? 

Exercise 8.2. Use thè time special form (Section 7.1) to experimentally mea- 
sure thè evaluation times for thè list-sort-best-first-let procedure. Do thè results 
match thè expected running times based on thè 0(n 2 ) asymptotic running time? 

You may fìnd it helpful to define a procedure that constructs a list containing 
n random elements. To generate thè random elements use thè built-in proce- 
dure random that takes one number as input and evaluates to a pseudorandom 
number between 0 and one less than thè value of thè input number. Be careful 
in your time measurements that you do not include thè time required to gener- 
ate thè input list. 

Exercise 8.3. Defìne thè list-find-best procedure using thè list- accumulate pro- 
cedure from Section 5.4.2 and evaluate its asymptotic running time. 

Exercise 8.4. [*] Define and analyze a list-sort-worst-last procedure that sorts 
by finding thè worst element first and putting it at thè end of thè list. 

8.1.2 Insertion Sort 

The list-sort-best-first procedure seems quite inefficient. For every output ele- 
ment, we are searching thè whole remaining list to find thè best element, but 
do nothing of value with all thè comparisons that were done to find thè best 
element. 

An alternate approach is to build up a sorted list as we go through thè elements. 
Insertion sort works by putting thè first element in thè list in thè right place in 
thè list that results from sorting thè rest of thè elements. 

First, we define thè list-insert-one procedure that takes three inputs: a compar- 
ison procedure, an element, and a List. The input List must be sorted according 
to thè comparison function. As output, list-insert-one produces a List consist- 
ing of thè elements of thè input List, with thè input element inserts in thè right 
place according to thè comparison function. 
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(define ( list-insert-one cf el p) ; requires: p is sorted by cf 
(if {nuli? p) { list el) 

(if {cf el {car p)) {cons el p) 

{cons {car p) {list-insert-one cf el {cdr p)))))) 



The running time for list-insert-one is in 0(w) where n is thè number of elements 
in thè input list. In thè worst case, thè input element belongs at thè end of thè 
list and it makes n recursive applications of list-insert-one. Each application 
involves Constant work so thè overall running time of list-insert-one is in0(w). 

To sort thè whole list, we insert each element into thè list that results from sort- 
ing thè rest of thè elements: 

(define ( list-sort-insert cf p) 

(if {nuli? p) nuli 

{list-insert-one cf {car p) {list-sort-insert cf ( cdr /?))))) 


Evaluating an application of list-sort-insert on a list of length n involves n recur- 
sive applications. The lengths of thè input lists in thè recursive applications are 
n — 1, n — 2, . . ., 0. Each application involves an application of list-insert-one 
which has linear running time. The average length of thè input list over all thè 
applications is approximately \, so thè average running time of thè list-insert- 
one applications is in 0(«). There are n applications of list-insert-one, so thè 
total running time is in 0(n 2 ). 


Exercise 8.5. We analyzed thè worst case running time of list-sort-insert above. 
Analyze thè best case running time. Your analysis should identify thè inputs for 
which list-sort-insert runs fastest, and describe thè asymptotic running time for 
thè best case input. 

Exercise 8.6. Both thè list-sort-best-first-sort and list-sort-insert procedures 
have asymptotic running times in 0(n 2 ). This tells us how their worst case run- 
ning times grow with thè size of thè input, but isn’t enough to knowwhich proce- 
dure is faster for a particular input. For thè questions below, use both analytical 
and empirical analysis to provide a convincing answer. 

a. How do thè actual running times of list-sort-best-first-sort and list-sort-insert 
on typical inputs compare? 

b. Are there any inputs for which list-sort-best-flrst is faster than list-sort-insert ? 

c. For sorting a long list of n random elements, how long does each procedure 
take? (See Exercise 8.2 for how to create a list of random elements.) 

8.1.3 Quicker Sorting 

Although insertion sort is typically faster than best-first sort, its running time is 
stili scales quadratically with thè length of thè list. If it takes 100 milliseconds 
(one tenth of a second) to sort a list containing 1000 elements using list-sort- 
insert, we expect it will take four (= 2 2 ) times as long to sort a list containing 
2000 elements, and a million times (= 1000 2 ) as long (over a day!) to sort a list 
containing one million (1000 * 1000) elements. Yet computers routinely need to 
sort lists containing many millions of elements (for example, consider process- 
ing credit card transactions or analyzing thè data collected by a super collider) . 
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The problem with our insertion sort is that it divides thè work unevenly into 
inserting one element and sorting thè rest of thè list. This is a very unequal 
division. Any sorting procedure that works by considering one element at a time 
and putting it in thè sorted position as is done by list-sort-find-best and list- 
sort-insert has a running time in Cl(n 2 ). We cannot do better than this with this 
strategy since there are n elements, and thè time required to figure out where 
each element goes is in Q(n). 

To do better, we need to either reduce thè number of recursive applications 
needed (this would mean each recursive cali results in more than one element 
being sorted), or reduce thè time required for each application. The approach 
we take is to use each recursive application to divide thè list into two approx- 
imately equal-sized parts, but to do thè division in such a way that thè results 
of sorting thè two parts can be combined directly to form thè result. We par- 
tition thè elements in thè list so that all elements in thè first part are less than 
(according to thè comparison function) all elements in thè second part. 

Our first attempt is to modify insert-one to partition thè list into two parts. This 
approach does not produce a better-than-quadratic time sorting procedure be- 
cause of thè inefficiency of accessing list elements; however, it leads to insights 
for producing a quicker sorting procedure. 

First, we define a list-extract procedure that takes as inputs a list and two num- 
bers indicating thè start and end positions, and outputs a list containing thè 
elements of thè input list between thè start and end positions: 

(define ( list-extract p start end) 

(if (= start 0) 

(if (= end 0) nuli 

( cons ( car p) ( list-extract ( cdr p) start (— end 1)))) 

( list-extract ( cdr p) (— start 1) (— end 1)))) 

The running time of thè list-extract procedure is in 0(n) where n is thè number 
of elements in thè input list. The worst case input is when thè value of end is thè 
length of thè input list, which means there will be n recursive applications, each 
involving a Constant amount of work. 

We use list-extract to define procedures for obtaining first and second halves of 
a list (when thè list has an odd number of elements, we put thè middle element 
in thè second half of thè list): 

(define ( list-flrst-half p) 

( list-extract p 0 {floor (/ ( list-length p) 2)))) 

(define ( list-second-half p) 

( list-extract p ( floor (/ ( list-length p) 2)) ( list-length p ) ) ) 

The list-flrst-half and list-second-half procedures use list-extract so their run- 
ning times are linear in thè number of elements in thè input list. 

The list-insert-one-split procedure inserts an element in sorted order by first 
splitting thè list in halves and then recursively inserting thè new element in thè 
appropriate half of thè list: 
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(define {lisi- i n s eri- o n e- split cf el p) ; requires: p is sorted by cf 
(if ( nuli ? p) { list el) 

(if [nuli? ( cdr p)) 

(if ( cf el ( car p)) ( cons el p) ( list ( car p) el)) 

(let {{front {list-first-half p)) ( back {list-second-half p))) 

(if {cf el {car back)) 

{list-append {list-insert-one-split cf el front) back) 

{list- append front ( list-insert-one-split cf el back))))))) 

In addition to thè normal base case when thè input list is nuli, we need a special 
case when thè input list has one element. If thè element to be inserted is before 
this element, thè output is produced using cons ; otherwise, we produce a list of 
thè first (only) element in thè list followed by thè inserted element. 

In thè recursive case, we use thè list-first-half and list-second-half procedures 
to split thè input list and bind thè results of thè first and second halves to thè 
front and back variables so we do not need to evaluate these expressions more 
than once. 

Since thè list passed to list-insert-one-split is required to be sorted, thè elements 
in front are all less than thè first element in back. Hence, only one comparison 
is needed to determine which of thè sublists contains thè new element: if thè 
element is before thè first element in back it is in thè first half, and we produce 
thè result by appending thè result of inserting thè element in thè front half with 
thè back half unchanged; otherwise, it is in thè second half, so we produce thè 
result by appending thè front half unchanged with thè result of inserting thè 
element in thè back half. 

To analyze thè running time of list-insert-one-split we determine thè number 
of recursive calls and thè amount of work involved in each application. We use 
n to denote thè number of elements in thè input list. Unlike thè other recur- 
sive list procedures we have analyzed, thè number of recursive applications of 
list-insert-one-split does not scale linearly with thè length of thè input list. The 
reason for this is that instead of using {cdr p) in thè recursive cali, list-insert-one- 
split passes in either thè front or back value which is thè result of ( first-half p) or 
( second-half p) respectively. The length of thè list produced by these procedures 
is approximately \ thè length of thè input list. With each recursive application, 
thè size of thè input list is halved. This means, doubling thè size of thè input list 
only adds one more recursive application. This means thè number of recursive 
calls is logarithmic in thè size of thè input. 

Recali that thè logarithm (log ; ,) of a number n is thè number x such that b x — n 
where b is thè base of thè logarithm. In computing, we most commonly en- 
counter logarithms with base 2. Doubling thè input value increases thè value of 
its logarithm base two by one: log 2 2n = 1 + log 2 n. Changing thè base of a loga- 
rithm from k to b changes thè value by thè Constant factor (see Section 7.3.1), so 
inside thè asymptotic operators a Constant base of a logarithm does not matter. 
Thus, when thè amount of work increases by some Constant amount when thè 
input size doubles, we write that thè growth rate is in 0(logn) without specify- 
ing thè base of thè logarithm. 

Each list-insert-one-split application applies list-append to a first parameter that 
is either thè front half of thè list or thè result of inserting thè element in thè front 
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half of thè list. In either case, thè length of thè list is approximately |. The run- 
ning time of list-append is in 0(m) where m is thè length of thè first input list. 
So, thè time required for each list-insert-one-split application is in 0(n) where 
n is thè length of thè input list to list-insert-one-split. 

The lengths of thè input lists to list-insert-one-split in thè recursive calls are ap- 
proximately j, |, g, . . ., 1, since thè length of thè list halves with each cali. The 
summation has log 2 n terms, and thè sum of thè list is n, so thè average length 
input is lo ” w . Hence, thè total running time for thè list-append applications in 

each application of list-insert-one-split is in 0(log 2 n x |o " - ) = 0(n). 

The analysis of thè applications of list-flrst-half and list-second-half is simi- 
lar: each requires running time in 0(m) where m is thè length of thè input list, 
which averages |o ” n where n is thè length of thè input list of list-insert-one-split. 
Hence, thè total running time for list-insert-one-split is in 0(n). 

The list-sort-insert-split procedure is identical to list-sort-insert (except for call- 
ing list-insert-one-split): 

(define ( list-sort-insert-split cf p) 

(if ( nuli ? p) nuli 

( list-insert-one-split cf ( car p) ( list-sort-insert-split cf {cdr /?))))) 

Similarly to list-sort-insert, list-sort-insert-split involves n applications of list- 
insert-one-split, and thè average length of thè input list is 2 . Since list-sort- 
insert-split involves 0(n) applications of list-insert-one-split with average in- 
put list length of |, thè total running time for list-sort-insert-split is in 0(n 2 ). 
Because of thè cost of evaluating thè list-append, list-flrst-half, and list-second- 
half applications, thè change to splitting thè list in halves has not improved thè 
asymptotic performance; in fact, because of all thè extra work in each applica- 
tion, thè actual running time is higher than it was for list-sort-insert. 

The problem with our list-insert-one-split procedure is that thè list-flrst-half 
and list-second-half procedures have to cdr down thè whole list to get to thè 
middle of thè list, and thè list-append procedure needs to walk through thè en- 
tire input list to put thè new element in thè list. All of these procedures have run- 
ning times that scale linearly with thè length of thè input list. To use thè splitting 
strategy effectively, we need is a way to get to thè middle of thè list quickly. With 
thè standard list representation this is impossible: it requires one cdr applica- 
tion to get to thè next element in thè list, so there is no way to access thè middle 
of thè list without using at least | applications of cdr. To do better, we need to 
change thè way we represent our data. The next subsection introduces such a 
structure; in Section 8.1.5 shows a way of sorting efficiently using lists directly 
by changing how we split thè list. 

8.1.4 BinaryTrees 

The data structure we will use is known as a sorted binary tree. While a list pro- 
vides Constant time procedures for accessing thè first element and thè rest of thè 
elements, a binary tree provides Constant time procedures for accessing thè root 
element, thè left side of thè tree, and thè right side of thè tree. The left and right 
sides of thè tree are themselves trees. So, like a list, a binary tree is a recursive 
data structure. 


sorted binary tree 
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Whereas we defined a List (in Chapter 5) as: 

A List is either (1) nuli or (2) a Pair whose second celi is a List. 
a Tree is defined as: 

A Tree is either (1) nuli or (2) a triple while first and third parts are both 
Tree s. 

Symbolically: 

Tree ::=> nuli 

Tree ::=> ( make-tree Tree Element Tree ) 


The malce-tree procedure can be defined using cons to package thè three inputs 
into a tree: 

(define ( make-tree left element right ) 

( cons element ( cons left right))) 

We define selector procedures for extracting thè parts of a non- nuli tree: 

(define ( tree-element tree) ( car tree)) 

(define (tree- le fi tree) ( car ( cdr tree))) 

(define (tree -right tree) ( cdr ( cdr tree))) 

The tree-left and tree-right procedures are Constant time procedures that evalu- 
ate to thè left or right subtrees respectively of a tree. 

In a sorted tree, thè elements are maintained in a sorted structure. All elements 
in thè left subtree of a tree are less than (according to thè comparison function) 
thè value of thè root element of thè tree; all elements in thè right subtree of a tree 
are greater than or equal to thè value of thè root element of thè tree (thè result 
of comparing them with thè root element is false) . For example, here is a sorted 
binary tree containing 6 elements using < as thè comparison function: 


7 



1 6 17 


The top node has element value 7, and its left subtree is a tree containing thè 
tree elements whose values are less than 7. The nuli subtrees are not shown. For 
example, thè left subtree of thè element whose value is 1 2 is nuli. Although there 
are six elements in thè tree, we can reach any element from thè top by following 
at most two branches. By contrast, with a list of six elements, we need five cdr 
operations to reach thè last element. 

depth The depili of a tree is thè largest number of steps needed to reach any node in 
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thè tree starting from thè root. The example tree has depth 2, since we can reach 
every node starting from thè root of thè tree in two or fewer steps. A tree of depth 
d can contain up to 2 d+1 — 1 elements. One way to see this is from this recursive 
defmition for thè maximum number of nodes in a tree: 


TreeNodes(d) 


1 : d = 0 

TreeNodes(d — 1) + 2 x TreeLeaves(d — 1) : d > 0 


A tree of depth zero has one node. Increasing thè depth of a tree by one means 
we can add two nodes for each leaf node in thè tree, so thè total number of nodes 
in thè new tree is thè sum of thè number of nodes in thè originai tree and twice 
thè number of leaves in thè originai tree. The maximum number of leaves in a 
tree of depth d is 2 d since each level doubles thè number of leaves. Hence, thè 
second equation simplihes to 

TreeNodes(d — 1) + 2 x 2 d_1 = TreeNodes(d — 1) + 2 d . 

The value of TreeNodes(d — 1) is 2 d ~ 1 + 2 d ~ 2 + . . . + 1 = 2 d — 1. Adding 2 d and 
2 d — 1 gives 2 d+1 — 1 as thè maximum number of nodes in a tree of depth d. 

Hence, a well-balanced tree containing n nodes has depth approximately log 2 n. 

A tree is well-balanced if thè left and right subtrees of all nodes in thè contain well-balanced 
nearly thè same number of elements. 


Procedures that are analogous to thè list-flrst-half , list-second-half , and list- 
append procedures that had linear running times for thè standard list represen- 
tation can all be implemented with Constant running times for thè tree repre- 
sentation. For example, tree-left is analogous to list-flrst-half and make-tree is 
analogous to list-append. 

The tree-insert-one procedure inserts an element in a sorted binary tree: 

(define ( tree-insert-one cf el tree ) 

(if ( nuli ? tree) [ make-tree nuli el nuli) 

(if [cf el ( tree-element tree)) 

[make-tree [tree-insert-one cf el [tree-left tree)) 

[tree-element tree) 

[tree -right tree)) 

[make-tree [tree-left tree) 

[tree-element tree) 

[tree-insert-one cf el [tree-right tree)))))) 


When thè input tree is nuli, thè new element is thè top element of a new tree 
whose left and right subtrees are nuli. Otherwise, thè procedure compares thè 
element to insert with thè element at thè top node of thè tree. If thè comparison 
evaluates to true, thè new element belongs in thè left subtree. The result is a tree 
where thè left tree is thè result of inserting this element in thè old left subtree, 
and thè element and right subtree are thè same as they were in thè originai tree. 
For thè alternate case, thè element is inserted in thè right subtree, and thè left 
subtree is unchanged. 


In addition to thè recursive cali, tree-insert-one only applies Constant time pro- 
cedures. If thè tree is well-balanced, each recursive application halves thè size 
of thè input tree so there are approximately log 2 n recursive calls. Hence, thè 
running time to insert an element in a well-balanced tree using tree-insert-one 
is in 0(log«). 
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Using tree-insert-one, we defìne list-to-sorted-tree, a procedure that takes a com- 
parison function and a list as its inputs, and outputs a sorted binary tree con- 
taining thè elements in thè input list. It inserts each element of thè list in turn 
into thè sorted tree: 

(define ( list-to-sorted-tree cf p) 

(if ( nuli ? p ) nuli 

( tree-insert-one cf ( car p) ( list-to-sorted-tree cf ( cdr p))))) 

Assuming well-balanced trees as above (we revisit this assumption later), thè 
expected running time of list-to-sorted-tree is in 0(n log n) where n is thè size of 
thè input list. There are n recursive applications of list-to-sorted-tree since each 
application uses cdr to reduce thè size of thè input list by one. Each application 
involves an application of tree-insert-one (as well as only Constant time proce- 
dures), so thè expected running time of each application is in ©(log n). Hence, 
thè total running time for list-to-sorted-tree is in 0(n log n). 

To use our list-to-sorted-tree procedure to perform sorting we need to extract 
a list of thè elements in thè tree in thè correct order. The leftmost element in 
thè tree should be thè fìrst element in thè list. Starting from thè top node, all 
elements in its left subtree should appear before thè top element, and all thè el- 
ements in its right subtree should followit. The tree-extract-elements procedure 
does this: 

(define ( tree-extract-elements tree) 

(if ( nuli ? tree ) nuli 

( list-append ( tree-extract-elements ( tree-left tree)) 

( cons ( tree-element tree) 

( tree-extract-elements ( tree-right tree)))))) 

The total number of applications of tree-extract-elements is between n (thè num- 
ber of elements in thè tree) and 3 n since there can be up to two nuli trees for each 
leaf element (it could never actually be 3 n, but for our asymptotic analysis it is 
enough to know it is always less than some Constant multiple of n). For each ap- 
plication, thè body applies list-append where thè fìrst parameter is thè elements 
extracted from thè left subtree. The end result of all thè list-append applications 
is thè output list, containing thè n elements in thè input tree. 

Hence, thè total size of all thè appended lists is at most n, and thè running time 
for all thè list-append applications is in 0(«). Since this is thè total time for all 
thè list-append applications, not thè time for each application of tree-extract- 
elements, thè total running time for tree-extract-elements is thè time for thè re- 
cursive applications, in 0(n), plus thè time for thè list-append applications, in 
0(n), which is in 0(n). 

Putting things together, we defìne list-sort-tree: 

(define ( list-sort-tree cf p) 

( tree-extract-elements ( list-to-sorted-tree cf p) ) ) 

The total running time for list-sort-tree is thè running time of thè list-to-sorted- 
tree application plus thè running time of thè tree-extract-elements application. 
The running time of list-sort-tree is in 0(nlogn) where n is thè number of el- 
ements in thè input list (in this case, thè number of elements in p), and thè 
running time of tree-extract-elements is in 0(n) where n is thè number of eie- 
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ments in its input list (which is thè result of thè list-to-sorted tree application, a 
list containing n elements where n is thè number of elements in p). 

Only thè fastest-growing term contributes to thè total asymptotic running tinte, 
so thè expected total running tinte for an application of list-sort-tree-insert to a 
list containing n elements is in &(n log n). This is substantially better than thè 
previous sorting algorithms which had running times in @(n 2 ) since logarithms 
grow far slower than their input. For example, if n is one million, n 2 is over 50,000 
times bigger than n log 2 n; if n is one billion, n 1 is over 33 million times bigger 
than n log 2 n since log 2 1000000000 is just under 30. 

There is no generai sorting procedure that has expected running time better 
than 0(n logn), so there is no algorithm that is asymptotically faster than list- 
sort-tree (in fact, it can be proven that no asymptotically faster sorting procedure 
exists). There are, however, sorting procedures that may have advantages such 
as how they use memory which may provide better absolute performance in 
some situations. 

Unbalanced Trees. Our analysis assumes thè left and right halves of thè tree 
passed to tree-insert-one having approximately thè same number of elements. 
If thè input list is in random order, this assumption is likely to be valid: each 
element we insert is equally likely to go into thè left or right half, so thè halves 
contain approximately thè same number of elements all thè way down thè tree. 
But, if thè input list is not in random order this may not be thè case. 

For example, suppose thè input list is already in sorted order. Then, each ele- 
ment that is inserted will be thè rightmost node in thè tree when it is inserted. 
For thè previous example, this produces thè unbalanced tree shown in Figure 8.1. 
This tree contains thè same six elements as thè earlier example, but because it is 
not well-balanced thè number of branches that must be traversed to reach thè 
deepest element is 5 instead of 2. Similarly, if thè input list is in reverse sorted 
order, we will have an unbalanced tree where only thè left branches are used. 

In these pathological situations, thè tree effectively becomes a list. The num- 
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ber of recursive applications of tree-insert-one needed to insert a new element 
will not be in 0(logn), but rather will be in 0(n). Hence, thè worst case run- 
ning tinte for list-sort-tree-insert is in 0(n 2 ) since thè worst case tinte for tree- 
insert-one is in 0(n) and there are 0(«) applications of tree-insert-one. The list- 
sort-tree-insert procedure has expected running tinte in 0(n log n) for randomly 
distributed inputs, but has worst case running tinte in 0(« 2 ). 


Exercise 8.7. Define a procedure binary-tree-size that takes as input a binary 
tree and outputs thè number of elements in thè tree. Analyze thè running tinte 
of your procedure. 


Myflrst task was to 
implement a library 
subroutine for a 
new fast method of 
internai sorting just 
invented by Shell. . . 

My boss and tutor, 
Pat Shackleton, was 
very pleased with 
my completed 
program. I then 
said timidly thatl 
thoughtlhad 
invented a sorting 
method that would 
usually runfaster 
than Shell sort, 
without taking 
much extra store. 
He bet me sixpence 
thatlhad not. 
Although my 
method was very 
dijfcult to explain, 
hefìnally agreed 
thatlhad won my 
bet. 

Sir Tony Hoare, The 
Emperor’s Old Clothes, 
1980 TuringAward 
Lecture. (Shell sort is a 
0 (h 2 ) sorting 
algorithm, somewhat 
similar to insertion 
sort.) 


Exercise 8.8. [*] Define a procedure binary-tree-depth that takes as input a bi- 
nary tree and outputs thè depth of thè tree. The running tinte of your procedure 
should not grow faster than linearly with thè number of nodes in thè tree. 

Exercise 8.9. [** Define a procedure binary-tree-balance that takes as input a 
sorted binary tree and thè comparison function, and outputs a sorted binary 
tree containing thè same elements as thè input tree but in a well-balanced tree. 
The depth of thè output tree should be no higher than log 2 n + 1 where n is thè 
number of elements in thè input tree. 

8.1.5 Quicksort 

Although building and extracting elements from trees allows us to sort with ex- 
pected time in &(n log n), thè Constant time required to build all those trees and 
extract thè elements from thè final tree is high. 

In fact, we can use thè same approach to sort without needing to build trees. 
Instead, we keep thè two sides of thè tree as separate lists, and sort them recur- 
sively. The key is to divide thè list into halves by vaine, instead of by position. 
The values in thè first half of thè list are all less than thè values in thè second half 
of thè list, so thè lists can be sorted separately. 

The list- quicksort procedure uses list-filter (from Example 5.5) to divide thè in- 
put list into sublists containing elements below and above thè comparison ele- 
ment, and then recursively sorts those sublists: 

(define ( list-quicksort cf p) 

(if ( nuli ? p) nuli 
( list-append 
( list-quicksort cf 

{list-filter (lambda [el) [cf el { car fi))) ( cdr p ) ) ) 

( cons ( car p) 

( list-quicksort cf 

{ list-filter (lambda ( el ) ( not {cf el {car fi)))) {cdr p))))))) 

This is thè famous quicksort algorithm that was invented by Sir C. A. R. (Tony) 
Hoare while he was an exchange student at Moscow State University in 1959. He 
was there to study probability theory, but also got a job working on a project to 
translate Russian into English. The translation depended on looking up words 
in a dictionary. Since thè dictionary was stored on a magnetic tape which could 
be read in order faster than if it was necessary to jump around, thè translation 
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could be done more quickly if thè words to translate were sorted alphabetically. 
Hoare invented thè quicksort algorithm for this purpose and it remains thè most 
widely used sorting algorithm. 


As with list-sort-tree-insert, thè expected running time for a randomly arranged 
list is in 0(n log n) and thè worst case running time is in 0(n 2 ). In thè expected 
cases, each recursive cali halves thè size of thè input list (since if thè list is ran- 
domly arranged we expect about half of thè list elements are below thè value of 
thè first elementi, so there are approximately log n expected recursive calls. 


Each cali involves an application of list-filter, which has running time in 0(m) 
where m is thè length of thè input list. At each cali depth, thè total length of thè 
inputs to all thè calls to list-filter is n since thè originai list is subdivided into 2 rf 
sublists, which together include all of thè elements in thè originai list. Hence, thè 
total running time is in 0(n log n) in thè expected cases where thè input list is 
randomly arranged. As with list-sort-tree-insert, if thè input list is not randomly 
rearranged it is possible that all elements end up in thè same partition. Hence, 
thè worst case running time of list- quicksort is stili in 0(n 2 ). 


Exercise 8.10. Estimate thè time it would take to sort a list of one million ele- 
ments using list- quicksort. 

Exercise 8.11. Both thè list- quicksort and list-sort-tree-insert procedures have 
expected running times in 0(w log n). Experimentally compare their actual run- 
ning times. 

Exercise 8. 12. What is thè best case input for list- quicksort" 1 . Analyze thè asymp- 
totic running time for list- quicksort on best case inputs. 

Exercise 8.13. [*] Instead of using binary trees, we could use ternary trees. 

A node in a ternary tree has two elements, a left element and a right element, 
where thè left element must be before thè right element according to thè com- 
parison function. Each node has three subtrees: left, containing elements be- 
fore thè left element; middle, containing elements between thè left and right 
elements; and right, containing elements after thè right element. Is it possible 
to sort faster using ternary trees? 


There are two ways 
of constructing a 
software design: one 
way is to make it so 
simple that there 
are obviously no 
defìciencies, and thè 
other way is to 
make it so 
complicated that 
there are no obvious 
defìciencies. The 
first method is far 
more diffìcult. It 
demands thè same 
skill, devotion, 
insight, and even 
inspiration as thè 
discovery ofthe 
simple physical 
laws which underlìe 
thè complex 
phenomena of 
nature. 

Sir Tony Hoare, The 
Emperor’s Old Clothes 
(1980 TuringAward 
Lecture) 


8.2 Searching 

In a broad sense, nearly all problems can be thought of as search problems. If 
we can defìne thè space of possible Solutions, we can search that space to find 
a correct solution. For example, to solve thè pegboard puzzle (Exploration 5.2) 
we enumerate all possible sequences of moves and search that space to find a 
winning sequence. For most interesting problems, however, thè search space is 
far too large to search through all possible Solutions. 

This section explores a few specific types of search problems. First, we consider 
thè simple problem of finding an element in a list that satisfìes some property. 
Then, we consider searching for an item in sorted data. Finally, we consider 
thè more specific problem of efficiently searching for documents (such as web 
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pages) that contain some target word. 

8.2.1 Unstructured Search 

Finding an item that satisfies an arbitrary property in unstructured data requires 
testing each element in turn until one that satisfies thè property is found. Since 
we have no more information about thè property or data, there is no way to 
more quickly find a satisfying element. 

The list-search procedure takes as input a matching function and a list, and out- 
puts thè first element in thè list that satisfies thè matching function or false if 
there is no satisfying element: 1 

(define ( list-search ef p) 

(if {nuli? p) false ; Not found 

(if (ef (car p)) (car p) (list-search ef (cdr p))))) 

For example, 

(list-search (lambda ( el ) (= 12 e/)) (intsto 10)) => false 
(list-search (lambda (el) (= 12 el)) (intsto 15)) => 12 
( list-search (lambda (el) (> el 12)) (intsto 15)) =>■ 13 

Assuming thè matching function has Constant running time, thè worst case run- 
ning time of list-search is linear in thè size of thè input list. The worst case is 
when there is no satisfying element in thè list. If thè input list has length n, there 
are n recursive calls to list-search , each of which involves only Constant time 
procedures. 

Without imposing more structure on thè input and comparison function, there 
is no more efficient search procedure. In thè worst case, we always need to test 
every element in thè input list before concluding that there is no element that 
satisfies thè matching function. 

8.2.2 Binary Search 

If thè data to search is structured, it may be possible to find an element that sat- 
isfies some property without examining all elements. Suppose thè input data 
is a sorted binary tree, as introduced in Section 8.1.4. Then, with a single com- 
parison we can determine if thè element we are searching for would be in thè 
left or right subtree. Instead of eliminating just one element with each appli- 
cation of thè matching function as was thè case with list-search, with a sorted 
binary tree a single application of thè comparison function is enough to exclude 
approximately half thè elements. 

The binaiy-tree-search procedure takes a sorted binary tree and two procedures 
as its inputs. The first procedure determines when a satisfying element has been 
found (we cali this thè ef procedure, suggesting equality). The second proce- 
dure, cf, determines whether to search thè left or right subtree. Since cf is used 
to traverse thè tree, thè input tree must be sorted by cf. 


1 If thè input list contains false as an element, we do not know when thè list-search result is false 
if it means thè element is not in thè list or thè element whose value is false satisfies thè property. An 
alternative would be to produce an errar if no satisfying element is found, but this is more awkward 
when list-search is used by other procedures. 
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(define ( binary-tree-search ef cf treé) ; requires: tree is sorted by cf 
(if ( nuli ? tree ) false 

(if (e/ ( tree-element tree)) ( tree-element tree) 

(if ( cf ( tree-element tree)) 

( binary-tree-search ef cf ( tree-left tree)) 

( binary-tree-search ef cf {tree -righi tree)))))) 

For example, we can search for a number in a sorted binary tree using — as thè 
equality function and < as thè comparison function: 

(define (binary-iree- number- search tree target) 

( binary-tree-search (lambda (et) (= target el)) 

(lambda (el) (< target el)) 
tree)) 

To analyze thè running time of binary-tree-search, we need to determine thè 
number of recursive calls. Like our analysis of list-sort-tree, we assume thè in- 
put tree is well-balanced. If not, all thè elements could be in thè right branch, 
for example, and binary-tree-search becomes like list-search in thè pathological 
case. 

If thè tree is well-balanced, each recursive cali approximately halves thè num- 
ber of elements in thè input tree since it passed in either thè left or right subtree. 
Hence, thè number of calls needed to reach a nuli tree is in 0(logn) where n is 
thè number of elements in thè input tree. This is thè depth of thè tree: binary- 
tree-search traverses one path from thè root through thè tree until either reach- 
ing an element that satisfies thè ef function, or reaching a nuli node. 

Assuming thè procedures passed as ef and cf have Constant running time, thè 
work for each cali is Constant except for thè recursive cali. Hence, thè total run- 
ning time for binary-tree-search is in ©(log n) where n is thè number of elements 
in thè input tree. This is a huge improvement over linear searching: with linear 
search, doubling thè number of elements in thè input doubles thè search time; 
with binary search, doubling thè input size only increases thè search time by a 
Constant. 

8.2.3 Indexed Search 

The limitation of binary search is we can only use is when thè input data is ai- 
ready sorted. What if we want to search a collection of documents, such as find- 
ing all web pages that contain a given word? The web visible to search engines 
contains billions of web pages most of which contain hundreds or thousands 
of words. A linear search over such a vast corpus would be infeasible: suppos- 
ing each word can be tested in 1 millisecond, thè time to search 1 trillion words 
would be over 30 years! 

Providing useful searches over large data sets like web documents requires find- 
ing a way to structure thè data so it is not necessary to examine all documents 
to perform a search. One way to do this is to build an index that provides a map- 
ping from words to thè documents that contain them. Then, we can build thè 
index once, store it in a sorted binary tree, and use it to perform all thè searches. 
Once thè index is built, thè work required to perform one search is just thè time 
it takes to look up thè target word in thè index. If thè index is stored as a sorted 
binary tree, this is logarithmic in thè number of distinct words. 
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Strings. We use thè built-in String datatype to represent documents and target 
words. A String is similar to a List, but specialized for representing sequences 
of characters. A convenient way to make a String it to just use doublé quotes 
around a sequence of characters. For example, "abcd" evaluates to a String con- 
taining four characters. 

The String datatype provides procedures for matching, ordering, and converting 
between Strings and Lists of characters: 

string= ?: String x String — » Boolean 

Outputs true if thè input Strings have exactly thè same sequence of charac- 
ters, otherwise false. 

string< ?: String x String — > Boolean 

Outputs true if thè first input String is lexicographically before thè second 
input String, otherwise false. 

string->list : String—» List 

Outputs a List containing thè characters in thè input String. 

list-> string: List — > String 

Outputs a String containing thè characters in thè input List. 

One advantage of using Strings instead of Lists of characters is thè built-in pro- 
cedures for comparing Strings; we could write similar procedures for Lists of 
characters, but lexicographic ordering is somewhat tricky to get right, so it is 
better to use thè built-in procedures. 

Building thè index. The entries in thè index are Pairs of a word represented 
as a string, and a list of locations where that word appears. Each location is a 
Pair consisting of a document identifier (for web documents, this is thè Uniform 
Resource Locator (URL) that is thè address of thè web page represented as a 
string) and a Number identifying thè position within thè document where thè 
word appears (we label positions as thè number of characters in thè document 
before this location) . 

To build thè index, we split each document into words and record thè position 
of each word in thè document. The first step is to define a procedure that takes 
as input a string representing an entire document, and produces a list of ( word 
. position ) pairs containing one element for each word in thè document. We 
define a word as a sequence of alphabetic characters; non-alphabetic characters 
including spaces, numbers, and punctuation marks separate words and are not 
included in thè index. 

The text-to-word-positions procedure takes a string as input and outputs a list 
of word-position pairs corresponding to each word in thè input. The inner pro- 
cedure, text-to-word-positions-iter, takes three inputs: a list of thè characters in 
thè document, a list of thè characters in thè current word, and a number repre- 
senting thè position in thè string where thè current word starts; it outputs thè list 
of ( word . position ) pairs. The value passed in as w can be nuli, meaning there 
is no current word. Otherwise, it is a list of thè characters in thè current word. 
A word starts when thè first alphabetic character is found, and continues until 
either thè first non-alphabetic character or thè end of thè document. We use thè 
built-in char-downcase procedure to convert all letters to their lowercase form, 
so KING, King, and king all correspond to thè same word. 
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(define ( text-to-word-positions s) 

(define ( text-to-word-positions-iter p w pos ) 

(if ( nuli ? p) 

(if ( nuli ? w ) nuli (lisi ( cons (list-> string w) pos))) 

(if ( not ( char-alphabetic ? ( car p) ) ) ; finished word 
(if ( nuli ? w) ; no current word 

( text-to-word-positions-iter ( cdr p) nuli (+ pos 1)) 

( cons ( cons (list-> string w) pos) 

( text-to-word-positions-iter ( cdr p) nuli 
(+ pos ( list-length w) 1)))) 

( text-to-word-positions-iter ( cdr p) 

( list-append w ( list ( char-downcase ( car p) ) ) ) 
pos)))) 

( text-to-word-positions-iter (slring-> lisi s) nuli 0)) 

The next step is to build an index from thè list of word-position pairs. To enable 
fast searching, we store thè index in a binary tree sorted by thè target word. The 
insert-into-index procedure takes as input an index and a word-position pair 
and outputs an index consisting of thè input index with thè input word-position 
pair added. 

The index is represented as a sorted binary tree where each element is a pair of a 
word and a list of thè positions where that word appears. Each word should ap- 
pear in thè tree only once, so if thè word-position pair to be added corresponds 
to a word that is already in thè index, thè position is added to thè corresponding 
list of positions. Otherwise, a new entry is added to thè index for thè word with 
a list of positions containing thè position as its only element. 

(define ( insert-into-index index wp) 

(if ( nuli ? index) 

( make-tree nuli ( cons ( car wp) ( list ( cdr wp))) nuli) 

(if (siringa ? ( car wp) ( car ( tree-element index))) 

( make-tree ( tree-left index) 

(i cons ( car ( tree-element index)) 

( list-append ( cdr ( tree-element index)) 

( list ( cdr wp)))) 

Uree- righi index)) 

(if ( string< ? ( car wp) ( car ( tree-element index))) 

( make-tree ( insert-into-index ( tree-left index) wp) 

( tree-element index) 

( tree-right index)) 

( make-tree ( tree-left index) 

( tree-element index) 

(. insert-into-index ( tree-right index) wp)))))) 

To insert all thè ( word . position) pairs in a list into thè index, we use insert-into- 
index to add each pair, passing thè resulting index into thè next recursive cali: 

(define ( insert-all-wps index wps) 

(if ( nuli ? wps) index 

( insert-all-wps ( insert-into-index index ( car wps)) ( cdr wps)))) 

To add all thè words in a document to thè index we use text-to-word-positions 
to obtain thè list of word-position pairs. Since we want to include thè document 
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identity in thè positions, we use list-map to add thè uri (a string that identifies 
thè document location) to thè position of each word. Then, we use insert-all- 
wps to add all thè word-position pairs in this document to thè index. The index- 
document procedure takes a document identiher and its text as a string, and 
produces an index of all words in thè document. 

(define ( i n d ex -doc urne ni uri text) 

( insert-all-wps 
nuli 

( list-map (lambda ( wp ) ( cons ( car wp ) ( cons uri ( cdr wp )))) 

( text- to - word-positions text ) ) ) ) 

We leave analyzing thè running time of index- document as an exercise. The im- 
portant point, though, is that it only has to be done once for a given set of doc- 
uments. Once thè index is built, we can use it to answer any number of search 
queries without needing to reconstruct thè index. 

Merging indexes. Our goal is to produce an index for a set of documents, not 
just a single document. So, we need a way to take two indexes produced by 
index- document and combine them into a single index. We use this repeat- 
edly to create an index of any number of documents. To merge two indexes, 
we combine their word occurrences. If a word occurs in both documents, thè 
word should appear in thè merged index with a position list that includes all thè 
positions in both indexes. If thè word occurs in only one of thè documents, that 
word and its position list should be included in thè merged index. 

(define ( merge-indexes di d2) 

(define ( merge-elements pi p2) 

(if ( nuli ? pi) p2 
(if ( nuli ? p2) pi 

(if ( string= ? ( car ( car pi)) ( car ( car p2))) 

0 cons ( cons ( car ( car pi)) 

( list-append ( cdr ( car pi)) ( cdr ( car p2)))) 

( merge-elements ( cdr pi) ( cdr p2))) 

(if ( string< ? ( car ( car pi)) ( car ( car p2))) 

[cons ( car pi) ( merge-elements ( cdr pi) p2)) 

( cons ( car p2) ( merge-elements pi ( cdr p2) ))))))) 

(lisi- Lo- so rted-t ree 

(lambda [el e2) ( string< ? ( car el) ( car e.2) ) ) 

[. merge-elements ( tree-extract-elements di) 

( tree-extract-elements d2))))))) 

To merge thè indexes, we first use tree-extract-elements to convert thè tree rep- 
resentations to lists. The inner merge-elements procedure takes thè two lists of 
word-position pairs and outputs a single list. 

Since thè lists are sorted by thè target word, we can perform thè merge effi- 
ciently. If thè first words in both lists are thè same, we produce a word-position 
pair that appends thè position lists for thè two entries. If they are different, we 
use string< ? to determine which of thè words belongs first, and include that el- 
ement in thè merged list. This way, thè two lists are kept synchronized, so there 
is no need to search thè lists to see if thè same word appears in both lists. 

Obtaining documents. To build a useful index for searching, we need some 
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documents to index. The web provides a useful collection of freely available 
documents. To read documents from thè web, we use library procedures pro- 
vided by DrRacket. 

This expression loads thè libraries for managing URLs and getting bles from thè 
network: {require {lib "uri. ss" "net")). One procedure this library defìnes is string- 
> uri, which takes a string as input and produces a representation of that string 
as a URL. A Uniform Resource Locator (URL) is a standard way to identify a doc- 
ument on thè network. The address bar in most web browsers displays thè URL 
of thè current web page. 

The full grammar for URLs is quite complex (see Exercise 2.14), but we will use 
simple web page addresses of thè form: 2 


URL ::=^ 

Domain : :=>- 
SubDomains ::=f> 
SubDomains ::=>- 
OptPath ::=>- 
OptPath ::=>- 
Path ::=>- 


http:// Domain OptPath 
Name SubDomains 
. Domain 
e 

Path 

e 

/ Name OptPath 


An example of a URL is http://www.whitehouse.gov/index.html. The http indicates 
thè HyperText Transfer Protocol, which prescribes how thè web client (browser) 
and server communicate with each other. The domain name is www.whitehouse. 
gov, and thè path name is /index. html (which is thè default page for most web 
servers). 

The library also defìnes thè get-pure-port procedure that takes as input a URL 
and produces a port for reading thè document at that location. The read-char 
procedure takes as input a port, and outputs thè first character in that port. It 
also has a side effect: it advances thè port to thè next character. We can use 
read-char repeatedly to read each character in thè web page of thè port. When 
thè end of thè file is reached, thè next application of read-char outputs a special 
marker representing thè end of thè file. The procedure eof-object? evaluates to 
true when applied to this marker, and false for all other inputs. 

The read- all- chars procedure takes a port as its input, and produces a list con- 
taining all thè characters in thè document associated with thè port: 

(define {read- all- chars port ) 

(let ((c ( read-char port))) 

(if ( eof-object ? c) nuli 

{cons c {read-all-chars port))))) 

Using these procedures, we define web-get, a procedure that takes as input a 
string that represents thè URL of some web page, and outputs a string repre- 
senting thè contents of that page. 

(define ( web-get uri) 

{list-> string ( read-all-chars {get-pure-port {string->url uri))))) 


2 We use Name to represent sequences of characters in thè domain and path names, although thè 
actual rules for valid names for each of these are different. 
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To make it easy to build an index of a set of web pages, we defìne thè index- 
pages procedure that takes as input a list of web pages and outputs an index of 
thè words in those pages. It recurses through thè list of pages, indexing each 
document, and merging that index with thè result of indexing thè rest of thè 
pages in thè list. 

(define (index- pages p) 

(if ( nuli ? p) nuli 

(. merge-indexes ( index-document (car p) ( web-get ( car p) ) ) 

( index-pages ( cdr p) ) ) ) ) 

We can use this to create an index of any set of web pages. For example, here 
we use Jeremy Hylton’s collection of thè complete works of William Shakespeare 
(http://shakespeare.mit.edu) to define shakespeare-index asan index of thè words 
used in all of Shakespeare’s plays. 

(define shakespeare-index 
( index-pages 
(, list-map 
(lambda (play) 

(. string-append "http://shakespeare.mit.edu/" play "/full.html ")) 

;; List of plays following thè site’s naming conventions. 

(list "allswell" "asyoulikeit" "comedy_errors" "cymbeline" "Ili" 

"measure" "merry.wives" "merchant" "midsummer" "much.ado" 

"pericles" "taming.shrew" "tempest" "troilus.cressida" "twelfth_night" 
"two.gentlemen" "winters_tale" "Ihenryiv" "2henryiv" "henryv" 

"Ihenryvi" "2henryvi" "3henryvi" "henryviii" "john" " richardii " 

"richardiii" "Cleopatra" "coriolanus" "hamlet" "julius_caesar" "lear" 

"macbeth" "othello" "romeo_juliet" "timori" "titus")))) 

Building thè index takes about two and a half hours on my laptop. It contains 
22949 distinct words and over 1.6 million word occurrences. Much of thè time 
spent building thè index is in constructing new lists and trees for every change, 
which can be avoided by using thè mutable data types we cover in thè next chap- 
ter. The key idea, though, is that thè index only needs to be built once. Once thè 
documents have been indexed, we can use thè index to quickly perform any 
search. 

Searching. Using an index, searching for pages that use a given word is easy 
and efficient. Since thè index is a sorted binary tree, we use hinary-tree-search 
to search for a word in thè index: 

(define (search-in-index index word) 

(binary- tree- search 

(lambda (el) (string=? word (car el))) ; first element of (word . position) 

(lambda (el) (string<? word (car el))) 

index)) 

As analyzed in thè previous section, thè expected running time of binary-tree- 
search is in 0(logn) where n is thè number of nodes in thè input tree. 3 The 
body of search-in-index applies binary-tree-search to thè index. The number of 
nodes in thè index is thè number of distinct words in thè indexed documents. 


3 Because of thè way merge-indexes is defined, we do not actually get this expected running time. 
See Exercise 8.17. 
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So, thè running time of search- in- index scales logarithmically with thè number 
of distinct words in thè indexed documents. Note that thè number and size of 
thè documents does not matter! This is why a search engine such as Google 
can respond to a query quickly even though its index contains many billions of 
documents. 

One issue we should be concerned with is thè running time of thè procedures 
passed into binary-tree-search. Our analysis of binary-tree-search assumes thè 
equality and comparison functions are Constant time procedures. Here, thè pro- 
cedures as string= ? and string< ?, which both have worst case running times 
that are linear in thè length of thè input string. As used here, one of thè inputs 
is thè target word. So, thè amount of work for each binary-tree-search recursive 
cali is in @(w) where w is thè length of word. Thus, thè overall running time of 
search-in-index is in 0(w log d) where w is thè length of word and d is thè num- 
ber of words in thè index. If we assume all words are of some maximum length, 
though, thè w term disappears as a Constant factor (that is, we are assuming 
zv < C for some Constant C. Thus, thè overall running time is in ©(log d). 

Here are some examples: 


> [search-in-index shakespeare-index "mathematics") 
("mathematics" 

("http://shakespeare.mit.edu/taming_shrew/full.html" . 26917) 
("http://shakespeare.mit.edu/taming_shrew/full.html" . 75069) 
("http://shakespeare.mit.edu/taming_shrew/full.html" . 77341)) 

> [search-in-index shakespeare-index "procedure") 
false 


Our search-in-index and index-pages procedures form thè beginnings of a search 
engine Service. A useful web search engine needs at least two more capabilities: 
a way to automate thè process of hnding documents to index, and a way to rank 
thè documents that contain thè target word by thè likelihood they are useful. 
The exploration at thè end of this section addresses these capabilities. 


The mathematics 
and thè 

metaphysics, Fall to 
them asyoufind 
your stomach serves 
you ; No proflt grows 
where is no pleasure 
ta’en: In brief, sir, 
study whatyou 
most affect. 

William Shakespeare, 
The Taming of 
theShrew 


Histogram. We can also use our index to analyze Shakespeare’s writing. The 
index-histogram procedure produces a list of thè words in an index sorted by 
how frequently they appear: 

(define [index-histogram index) 

[list-quicksort 

(lambda [el e2 ) (> [cdr el ) [cdr e2) ) ) 

[list-map (lambda [el] [cons [car el) [length [cdr el)))) 
[tree-extract-elements index)))) 


The expression, 

[list-fllter (lambda [entry) (> string-length [car entiy) 5)) 

[index-histogram shakespeare-index)) 

evaluates to a list of Shakespeare’s favorite 6-letter and longer words along with 
thè number of times they appear in thè corpus (thè first two entries are from 
their use in thè page formatting) : 
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(("blockquote" . 63345) ("speech" . 31099) 

("should" . 1557) ("father" . 1086) ("exeunt" . 1061) 
("master" . 861) ("before" . 826) ("mistress" . 787) 

. . . ("brother" . 623) 

. . . ("daughter" . 452) 

. . . ("mother" . 418) 

. . . ("mustardseed" . 13) 

. . . ("excrement" . 5) 

. . . ("zwaggered" . 1)) 


Exercise 8.14. Define a procedure for finding thè longest word in a document. 
Analyze thè running time of your procedure. 

Exercise 8.15. Produce a list of thè words in Shakespeare’s plays sorted by their 
length. 

Exercise 8.16. *] Analyze thè running time required to build thè index. 

a. Analyze thè running time of thè text-to-word-positions procedure. Use n to 
represent thè number of characters in thè input string, and w to represent 
thè number of distinct words. Be careful to clearly state all assumptions on 
which your analysis relies. 

b. Analyze thè running time of thè insert-into-index procedure. 

c. Analyze thè running time of thè index-document procedure. 

d. Analyze thè running time of thè merge-indexes procedure. 

e. Analyze thè overall running time of thè ìndex-pages procedure. Your result 
should describe how thè running time is impacted by thè number of docu- 
ments to index, thè size of each document, and thè number of distinct words. 

Exercise 8.17. *] The search-in-index procedure does not actually have thè ex- 
pected running time in 0(logrc) (where w is thè number of distinct words in 
thè index) for thè Shakespeare index because of thè way it is built using merge- 
indexes. The problem has to do with thè running time of thè binary tree on 
pathological inputs. Explain why thè input to list-to-sorted-tree in thè merge- 
indexes procedure leads to a binary tree where thè running time for searching is 
in 0(ie). Modify thè merge-indexes definition to avoid this problem and ensure 
that searches on thè resulting index run in 0(log w). 

Exercise 8.18. [**] The site http://www.speechwars.com provides an interesting 
way to view politicai speeches by looking at how thè frequency of thè use of 
different words changes over time. Use thè index-histogram procedure to build 
a historical histogram program that takes as input a list of indexes ordered by 
time, and a target word, and output a list showing thè number of occurrences of 
thè target word in each of thè indexes. You could use your program to analyze 
how Shakespeare’s word use is different in tragedies and comedies or to compare 
Shakespeare’s vocabulary to Jefferson’s. 
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Exploration 8.1: Searching thè Web 


In addition to fast indexed search, web search engines have to solve two prob- 
lems: (1) find documents to index, and (2) identify thè most important docu- 
ments that contain a particular search term. 

For our Shakespeare index example, we manually found a list of interesting doc- 
uments to index. This approach does not scale well to indexing thè World Wide 
Web where there are trillions of documents and new ones are created all thè 
time. For this, we need a web crawler. 

A web crawler hnds documents on thè web. Typical web crawlers start with a set 
of seed URLs, and then find more documents to index by following thè links on 
those pages. This proceeds recursively: thè links on each newly discovered page 
are added to thè set of URLs for thè crawler to index. To develop a web crawler, 
we need a way to extract thè links on a given web page, and to manage thè set of 
pages to index. 

a. [*] Dehne a procedure extract-links that takes as input a string represent- 
ing thè text of a web page and outputs a list of all thè pages linked to from 
this page. Linked pages can be found by searching for anchor tags on thè 
web page. An anchor tag has thè form: 4 <a href=target>. (The text-to-word- 
positions procedure may be a helpful starting point for defming extract-links.) 

b. Define a procedure crawl-page that takes as input an index and a string rep- 
resenting a URL. As output, it produces a pair consisting of an index (that is 
thè result of adding all words from thè page at thè input URL to thè input 
index) and a list of URLs representing all pages linked to by thè crawled page. 

c. [*★] Define a procedure crawl- web that takes as input a list of seed URLs and 
a Number indicating thè maximum depth of thè crawl. It should output an 
index of all thè words on thè web pages at thè locations given by thè seed 
URLs and any page that can be reached from these seed URLs by following 
no more than thè maximum depth number of links. 

For a web search engine to be useful, we don’t want to just get a list of all thè 
pages that contain some target word, we want thè list to be sorted according to 
which of those pages are most likely to be interesting. Selecting thè best pages 
for a given query is a challenging and important problem, and thè ability to do 
this well is one of thè main things that distinguishes web search engines. Many 
factors are used to rank pages including an analysis of thè text on thè page itself, 
whether thè target word is part of a title, and how recently thè page was updated. 

The best ways of ranking pages also consider thè pages that link to thè ranked 
page. If many pages link to a given page, it is more likely that thè given page is 
useful. This property can also be defìned recursively: a page is highly ranked if 
there are many highly- ranked pages that link to it. 

The ranking System used by Google is based on this formula: 


R(u) 


y R( v ) 

k w 


web crawler 


The rank assigned 
to a document is 
calculated from thè 
ranks of documents 
citing it. In 
addition, thè rank 
ofa document is 
calculated from a 
Constant 
representing thè 
probability that a 
browser through thè 
database will 
randomly jump to 
thè document. The 
method is 
particularly useful 
in enhancing thè 
performance of 
search engine 
resultsfor 
hypermedia 
databases, such as 
thè world wide web, 
whose documents 
have a large 
variation in quality. 
United States Patent 
#6,285,999, September 
2001. (Inventar: 
Lawrence Page, 
Assignee: Stanford 
University) 


4 Not all links match this structure exactly, so this may miss some of thè links on a page. 
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where L u is thè set of web pages that contain links to thè target page u and L(v) 
is thè number of links on thè page v (thus, thè value of a link from a page con- 
taining many links is less than thè value of a link from a page containing only a 
few links). The value R(u) gives a measure of thè ranking of thè page identifìed 
by li, where higher values indicate more valuable pages. 

The problem with this formula is that is is circular: there is no base case, and no 
way to order thè web pages to compute thè correct rank of each page in turn, 
since thè rank of each page depends on thè rank of thè pages that link to it. 

relaxation One way to approximate equations like this one is to use relaxation. Relaxation 
obtains an approximate solution to some Systems of equations by repeatedly 
evaluating thè equations. To estimate thè page ranks for a set of web pages, we 
initially assume everypage has rank 1 and evaluate R(u ) for all thè pages (using 
thè value of 1 as thè rank for every other page). Then, re-evaluate thè R(u) values 
using thè resulting ranks. A relaxation keeps repeating until thè values change 
by less than some threshold amount, but there is no guarantee how quickly this 
will happen. For thè page ranking evaluation, it may be enough to decide on 
some fixed number of iterations. 

d. [**] Define a procedure, web-link-graph, that takes as input a set of URLs 
and produces as output a graph of thè linking structure of those documents. 
The linking structure can be represented as a List where each element of thè 
List is a pair of a URL and a list of all thè URLs that include a link to that URL. 
The extract-links procedure from thè previous exploration will be useful for 
determining thè link targets of a given URL. 

e. [*] Define a procedure that takes as input thè output of web-link-graph and 
outputs a preliminary ranking of each page that measures thè number of 
other pages that link to that page. 

f. [**] Refine your page ranking procedure to weight links from highly-ranked 
pages more heavily in a page’s rank by using a algorithm. 

g. [* * **] Come up with a cool name, set up your search engine as a web Ser- 
vice, and attract more than 0.001% of all web searches to your site. 


8.3 Summary 

The focus of Part II has been on predicting properties of procedures, in particu- 
lar how their running time scales with thè size of their input. This involved many 
encounters with thè three powerful ideas introduced in Section 1.4: recursive 
definitions, universality, and abstraction. The simple Turing Machine model is 
a useful abstraction for modeling nearly all conceivable computing machines, 
and thè few simple operations it defines are enough for a universal computer. 
Actual machines use thè digitai abstraction to allow a continuous range of volt- 
ages to represent just two values. The asymptotic operators used to describe 
running times are also a kind of abstraction — they allow us to represent thè set 
of infinitely many different functions with a compact notation. 

In Part III, we will see many more recursive definitions, and extend thè notion of 
recursive definitions to thè language interpreter itself. We change thè language 
evaluation rules themselves, and see how different evaluation rules enable dif- 
ferent ways of expressing computations. 
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Mutation 


Faced with thè choice between changing one's mìnd and proving that there is no 
need to do so, almost everyone gets busy on thè proof. 

John Kenneth Galbraith 

The subset of Scheme we have used until this chapter provides no means to 
change thè value associated with a name. This enabled very simple evaluation 
rules for names, as well as allowing thè substitution model of evaluation. Since 
thè value associated with a name was always thè value it was defined as, no com- 
plex evaluation rules are needed to determine thè value associated with a name. 

This chapter introduces special forms known as mutators that allow programs 
to change thè value in a given place. Introducing mutation does not change 
thè computations we can express — every computation that can be expressed 
using mutation could also be expressed using thè only purely functional subset 
of Scheme from Chapter 3. It does, however, make it possible to express cer- 
tain computations more efficiently and clearly than could be done without it. 
Adding mutation is not free, however; reasoning about thè value of expressions 
becomes much more complex. 

9.1 Assignment 

The set! (pronounced “set-bang!”) special form associates a new value with an 
already defined name. The exclamation point at thè end of set! follows a naming 
convention to indicate that an operation may mutate state. A set expression is 
also known as an assignment. It assigns a value to a variable. 

The grammar rule for assignment is: 

Expression ::=>• Assignment 
Assignment ::=> (set! Name Expression ) 


The evaluation rule for an assignment is: 

Evaluation Rule 7: Assignment. To evaluate an assignment, evaluate thè 
expression, and replace thè value associated with thè name with thè value 
of thè expression. An assignment has no value. 

Assignments do not produce output values, but are used for their side effects. 
They change thè value of some state (namely, thè value associated with thè name 
in thè set expression), but do not produce an output. 

Here is an example use of set!: 


mutators 


assignment 
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> (define num 200) 

> num 
200 

> (set! num 1 50) 

> (set! num 1120) 

> num 
1120 

Begin expression. Since assignments do not evaluate to a value, they are often 
used inside a begin expression. A begin expression is a special form that eval- 
uates a sequence of expressions in order and evaluates to thè value of thè last 
expression. 

The grammar rule for thè begin expression is: 

Expression BeginExpression 

BeginExpression ::=>• (begin MoreExpressions Expression ) 


The evaluation rule is: 

Evaluation Rule 8: Begin. To evaluate a begin expression, 

(begin Expression i Expression . . . Expression k ) 

evaluate each subexpression in order from left to right. The value of thè 
begin expression is thè value of thè last subexpression, Expression k . 

The values of all thè subexpressions except thè last one are ignored; these subex- 
pressions are only evaluated for their side effects. 

The begin expression must be a special form. It is not possible to define a pro- 
cedure that behaves identically to a begin expression since thè application rule 
does not specify thè order in which thè operand subexpressions are evaluated. 

The definition syntax for procedures includes a hidden begin expression. 

(define (Alarne Parameters ) MoreExpressions Expression ) 

is an abbreviation for: 

(define Alarne 

(lambda ( Parameters ) (begin MoreExpressions Expression ))) 

The let expression introduced in Section 8.1.1 also includes a hidden begin ex- 
pression. 

(let ((Alarne! Expression ) (Alame 2 Expression 2 ) 

■ • ■ (Alame/ f Expression )) 

MoreExpressions Expression ) 

is equivalent to thè application expression: 

((lambda (Alamei Name 2 . . . Alarne^) 

(begin MoreExpressions Expression )) 

Expression Expression ■ ■ ■ Expression ) 
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9.2 Impact of Mutation 

Introducing assignment presents many complications for our programming model. 
It invalidates thè substitution model of evaluation introduced in Section 3.6.2 
and found satisfactory until this point. All thè procedures we can define without 
using mutation behave almost like mathematical functions — every time they are 
applied to thè same inputs they produce thè same output. 1 Assignments allow 
us to define non-functional procedures that produce different results for differ- 
ent applications even with thè same inputs. 


Example 9.1: Counter 


Consider thè update-counter! procedure: 

(define ( update-counter !) 

(set! counter (+ counter 1)) 
counter) 

To use update-counter!, we must first define thè counter variable it uses: 

(define counter 0) 

Every time ( update-counter !) is evaluated thè value associated with thè name 
counter is increased by one and thè result is thè new value of counter. Because 
of thè hidden begin expression in thè definition, thè (set! counter (+ counter 1 )) 
is always evaluated first, followed by counter which is thè last expression in thè 
begin expression so its value is thè value of thè procedure. Thus, thè value of 
( update-counter !) is 1 thè first time it is evaluated, 2 thè second time, and so on. 

The substitution model of evaluation doesn’t make any sense for this evaluation: 
thè value of counter changes during thè course of thè evaluation. Even though 
( update-counter !) is thè same expression, every time it is evaluated it evaluates 
to a different value. 

Mutation also means some expressions have undetermined values. Consider 
evaluating thè expression (+ counter [update-counter!)). The evaluation rule 
for thè application expression does not specify thè order in which thè operand 
subexpressions are evaluated. But, thè value of thè name expression counter 
depends on whether it is evaluated before or after thè application of update- 
counter! is evaluated! 

The meaning of thè expression is ambiguous since it depends on thè order in 
which thè subexpressions are evaluated. If thè second subexpression, counter, 
is evaluated before thè third subexpression, [update-counter!), thè value of thè 
expression is 1 thè first time it is evaluated, and 3 thè second time it is evaluated. 
Alternately, but stili following thè evaluation rules correctly, thè third subexpres- 
sion could be evaluated before thè second subexpression. With this ordering, 
thè value of thè expression is 2 thè first time it is evaluated, and 4 thè second 
time it is evaluated. 


^bservant readers should notice that we have already used a few procedures that are not func- 
tions including thè printing procedures from Section 4.5.1, and random and read-char from thè 
previous chapter. 
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9.2.1 Names, Places, Frames, and Environments 

Because assignments can change thè value associateci with a name, thè order in 
which expressions are evaluated now matters. As a result, we need to revisit sev- 
eral of our other evaluation rules and change thè way we think about processes. 


Since thè value associated with a name can now change, instead of associating a 
place value directly with a name we use a name as a way to identify a place. A place has 
a name and holds thè value associated with that name. With mutation, we can 
change thè value in a place; this changes thè value associated with thè place’s 
frame name. A frame is a collection of places. 

environment An environment is a pair consisting of a frame and a pointer to a parent envi- 
ronment. A special environment known as thè global environment has no par- 
ent environment. The global environment exists when thè interpreter starts, 
and is maintained for thè lifetime of thè interpreter. Initially, thè global envi- 
ronment contains thè built-in procedures. Names defined in thè interactions 
buffer are placed in thè global environment. Other environments are created 
and destroyed as a program is evaluated. Figure 9.1 shows some example envi- 
ronments, frames, and places. 

Every environment has a parent environment except for thè global environ- 
ment. All other environments descend from thè global environment. Hence, 
if we start with any environment, and continue to follow its parent pointers we 
always eventually reach thè global environment. 


The key change to our evaluation model is that whereas before we could eval- 
uate expressions without any notion of where they are evaluated, once we in- 
troduce mutation, we need to consider thè environment in which an expression 
is evaluated. An environment captures thè current state of thè interpreter. The 
value of an expression depends on both thè expression itself, and on thè envi- 
ronment in which it is evaluated. 


Global Environment 


counter r o J 


update-counter! 




Environment B 


|| JC88 ;.)| 


environment: parameters; () 

body: (begin (set! counter (+ counter 1)) 
counter) 


Figure 9.1. Sample environments. 

The global environment contains a frame with three names. Each name has an associated place 
that contains thè value associated with that name. The value associated with counter is thè 
currently 0. The value associated with set-counter! is thè procedure we defined in Example 9.1. 
A procedure is characterized by its parameters, body code, and a pointer to thè environment in 
which it will be evaluated. 
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9.2.2 Evaluation Rules with State 

Introducing mutation requires us to revise thè evaluation rute for names, thè 
definition rule, and thè application rule for constructed procedures. All of these 
rules must be adapted to be more precise about how values are associated with 
names by using places and environments. 

Names. The new evaluation rule for a name expression is: 

Stateful Evaluation Rule 2: Names. To evaluate a name expression, 
search thè evaluation environment’s frame for a place with a name that 
matches thè name in thè expression. If such a place exists, thè value of 
thè name expression is thè value in that place. Otherwise, thè value of 
thè name expression is thè result of evaluating thè name expression in 
thè parent environment. If thè evaluation environment has no parent, 
thè name is not dehned and thè name expression evaluates to an error. 

For example, to evaluate thè value of thè name expression x in Environment B 
in Figure 9.1, we first look in thè frame of Environment B for a place named x. 
Since there is no place named x in that frame, we follow thè parent pointer to 
Environment A, and evaluate thè value of thè name expression in Environment 
A. Environment A’s frame contains a place named x that contains thè value 7, so 
thè value of evaluating x in Environment B is 7. 

The value of thè same expression in thè Global Environment is 3 since that is thè 
value in thè place named x in thè Global Environment’s frame. 

To evaluate thè value of y in Environment A, we first look in thè frame in Envi- 
ronment A for a place named y. Since no y place exists, evaluation continues by 
evaluating thè expression in thè parent environment, which is thè Global Envi- 
ronment. The Global Environments frame does not contain a place named y, 
and thè global environment has no parent, so thè name is undehned and thè 
evaluation results in an error. 

Definition. The revised evaluation rule for a definition is: 

Stateful Definition Rule. A definition creates a new place with thè defi- 
nition’s name in thè frame associated with thè evaluation environment. 

The value in thè place is value of thè definition’s expression. If there is ai- 
ready a place with thè name in thè current frame, thè definition replaces 
thè old place with a new place and value. 

The rule for redefinitions means we could use define in some situations to mean 
something similar to set!. The meaningis different, though, since an assignment 
finds thè place associated with thè name and puts a new value in that place. 
Evaluating an assignment follows thè Stateful Evaluation Rule 2 to find thè place 
associated with a name. Hence, (define Name Expression ) has a different mean- 
ing from (set! Name Expression ) when there is no place named Name in thè 
current execution environment. To avoid this confusion, only use define for thè 
first definition of a name and always use set! when thè intent is to change thè 
value associated with a name. 

Application. The final rule that must change because of mutation is thè ap- 
plication rule for constructed procedures. Instead of using substitution, thè 
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new application rule creates a new environment with a frame containing places 
named for thè parameters. 

Stateful Application Rule 2: Constructed Procedures. To apply a con- 
structed procedure: 

1. Construct a new environment, whose parent is thè environment 
of thè applied procedure. 

2. For each procedure parameter, create a place in thè frame of thè 
new environment with thè name of thè parameter. Evaluate each 
operand expression in thè environment or thè application and ini- 
tialize thè value in each place to thè value of thè corresponding 
operand expression. 

3. Evaluate thè body of thè procedure in thè newly created environ- 
ment. The resulting value is thè value of thè application. 

Consider evaluating thè application expression ( lugger 3 4) where bigger is thè 
procedure from Example 3.3: (define ( bigger a b) (if (> ab) a b) ) ) . 

Evaluating an application of bigger involves following thè Stateful Application 
Rule 2. First, create a new environment. Since bigger was defìned in thè global 
environment, its environment pointer points to thè global environment. Hence, 
thè parent environment for thè new environment is thè global environment. 

Next, create places in thè new environment’s frame named for thè procedure 
parameters, a and b. The value in thè place associated with a is 3, thè value of 
thè fìrst operand expression. The value in thè place associated with b is 4. 

The final step is to evaluate thè body expression, (if (> ab) ab), in thè newly cre- 
ated environment. Figure 9.2 shows thè environment where thè body expression 
is evaluated. The values of a and b are found in thè application environment. 


Global Environment 



Application Env 1 ® 

a( 3 ) 


bc 4 m 




(if (> ab) a b ) 


environment:! parameters: ( a b) 
.body: (if (> a b) a b ) 


Figure 9.2. Environment created to evaluate ( bigger 3 4). 


The new application rule becomes more interesting when we consider proce- 
dures that create new procedures. For example, make-adder takes a number as 
input and produces as output a procedure: 

(define ( make-adder v ) (lambda (n) (+ ri u))) 
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The environment that results from evaluating (define ine ( make-adder 1)) is 
shown in Figure 9.3. The name ine has a value that is thè procedure resulting 
from thè application of ( make-adder 1). To evaluate thè application, we follow 
thè application rule above and create a new environment containing a frame 
with thè parameter name, ine, and its associated operand value, 1 . 


Global Environment 





Application Env 1 


vC 


T 




env:" params: (v) 
body: (lambda ( n ) (+ n v)) 


env:l params: (n) 
body: (+ n v) 


Figure 9.3. Environment after evaluating (define ine ( make-adder 1 )). 


The result of thè application is thè value of evaluating its body in this new en- 
vironment. Since thè body is a lambda expression, it evaluates to a procedure. 
That procedure was created in thè execution environment that was created to 
evaluate thè application of make-adder, hence, its environment pointer points 
to thè application environment which contains a place named ine holding thè 
value 1 . 

Next, consider evaluating ( ine 149). Figure 9.4 illustrates thè environment for 
evaluating thè body of thè ine procedure. The evaluation creates a new envi- 
ronment with a frame containing thè place n and its associated value 149. We 
evaluate thè body of thè procedure, (+ n v), in that environment. The value of 
n is found in thè execution environment. The value of v is not found there, so 
evaluation continues by looking in thè parent environment. It contains a place 
v containing thè value 1 . 


Exercise 9.1. Devise a Scheme expression that could have four possible values, 
depending on thè order in which application subexpressions are evaluated. 

Exercise 9.2. Draw thè environment that results after evaluating: 

> (define alpha 0) 

> (define beta 1 ) 

> (define update-beta! (lambda () (set! beta (+ alpha 1))) 

> (set! alpha 3) 

> ( update-beta !) 

> (set! alpha 4) 
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9.3. Mutable Pairs and Lists 


Global Envlronment 



Figure 9.4. Environment for evaluating thè body of ( ine 1 49). 


Exercise 9.3. Draw thè environment that results after evaluating thè following 
expressions, and explain what thè value of thè final expression is. (Hint: first, 
rewrite thè let expression as an application.) 

> (define ( make-applier proc ) (lambda (x) ( proc x)) 

> (define p ( make-applier (lambda (x) (let ((x2)) x)))) 

> (p 4) 


9.3 Mutable Pairs and Lists 

immutable The Pair datatype introduced in Chapter 5 is immutable. This means that once 
a Pair is created, thè values in its cells cannot be changed. 2 

The MutablePair datatype is a mutable pair. A MutablePair is constructed using 
meons, which is similar to cons but produces a MutablePair. The parts of a Mu- 
tablePair can be extracted using thè mear and medr procedures, which behave 
analogously to thè car and cdr procedures. A MutablePair is a distinct datatype 
from a Pair; it is an error to apply car to a MutablePair, or to apply mear to an 
immutable Pair. 

The MutablePair datatype also provides two procedures that change thè values 
in thè cells of a MutablePair: 

set-mear !: MutablePair x Value — >• Void 

Replaces thè value in thè first celi of thè MutablePair with thè value of thè 
second input. 

set-mcdr!\ MutablePair x Value — > Void 

Replaces thè value in thè second celi of thè MutablePair with thè value of 
thè second input. 

2 The mutability of standard Pairs is quite a controversial issue. In most Scheme implementations 
and thè standard definition of Scheme, a standard cons pair is mutable. But, as we will see later in 
thè section, mutable pairs cause lots of problems. So, thè designers of DrRacket decided for Version 
4.0 to make thè standard Pair datatype immutable and to provide a MutablePair datatype for use 
when mutation is needed. 






Chapter9. Mutation 


187 


The Void result type indicates that set-mcar! and set-mcdr! produce no output. 

Here are some interactions using a MutablePair: 

> (define pair ( mcons 1 2)) 

> [set-mcar! pair 3) 

> pair 
(3.2) 

> [set-mcdr! pair A) 

> pair 
(3.4) 

The set-mcdr! procedure allows us to create a pair where thè second celi of thè 
pair is itself: [set-mcdr! pair pair). This produces thè rather frightening object 
shown in Figure 9.5. Every time we apply mcdr to pair, we get thè same pair as 



Figure 9.5. Mutable pair created by evaluating [set-mcdr! pair pair). 


thè output. Hence, thè value of [mcar [mcdr [mcdr [mcdr pair)))) is 3. 

We can also create objects that combine mutable and immutable Pairs. For ex- 
ample, (define mstruct [cons [mcons 1 2) 3)) defines mstruct as an immutable 
Pair containing a MutablePair in its first celi. Since thè outer Pair is immutable, 
we cannot change thè objects in its cells. Thus, thè second celi of mstruct always 
contains thè value 3. We can, however, change thè values in thè cells of thè mu- 
table pair in its first celi. For example, [set-mcar! [car mstruct) 7) replaces thè 
value in thè first celi of thè MutablePair in thè first celi of mstruct. 

Mutable Lists. As we used immutable Pairs to build immutable Lists, we can 
use MutablePairs to construct MutableLists. A MutableList is either nuli or a Mu- 
tablePair whose second celi contains a MutableList. 

The MutableList type is defined by a library. To use it, evaluate thè following 
expression: [require racket/mpair) . All of thè examples in this chapter assume 
this expression has been evaluated. This library defines thè mlist procedure 
that is similar to thè list procedure, but produces a MutableList instead of an 
immutable List. For example, ( mlist 1 2 3) produces thè structure shown in Fig- 
ure 9.6. Each no de in thè list is a MutablePair, so we can use thè set-mcar! and 




Figure 9.6. MutableList created by evaluating [mlist 12 3). 


set-mcdr! procedures to change thè values in thè cells. 

> (define mi [mlist 1 2 3)) 

> [set-mcar! [mcdr mi) 5) 

> [set-mcar! [mcdr [mcdr mi)) 0) 

> mi 

{15 0}; DrRacket denotes MutableLists using curly brackets. 
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9.4. Imperative Programming 


Many of thè list procedures from Chapter 5 can be directly translated to work on 
mutable lists. For example, we can define mlist-length as: 

(define ( mlist-length ni) 

(if ( nuli ? ni) 0 (+ 1 ( mlist-length ( mcdr ni))))) 

As shown in Exercise 9.4, though, we need to be careful when using mcdr to 
recurse through a MutableList since structures created with MutablePairs can 
include circular pointers. 


Exercise 9.4. What is thè value of ( mlist-length pair) for thè pair shown in Fig- 
ure 9.5? 

Exercise 9.5. [*] Define a mpair- circular? procedure that takes a MutablePair as 
its input and outputs true when thè input contains a cycle and false otherwise. 


9.4 Imperative Programming 

imperative Mutation enables a style of programming known as imperative programming. 
programming whereas functional programming is concerned with defining procedures that 
can be composed to solve a problem, imperative programming is primarily con- 
cerned with modifying state in ways that lead to a state that provides a solution 
to a problem. 

The main operation in function programming is application. A functional pro- 
gram applies a series of procedures, passing thè outputs of one application as 
thè inputs to thè next procedure application. With imperative programming, thè 
primary operation is assignment (performed by set!, set-mcar!, and set-mcdr! in 
Scheme; but typically by an assignment operator, often : = or =, in languages de- 
signed for imperative programming such as Pascal, Algol60, Java, and Python). 

The next subsection presents imperative-style versions of some of thè proce- 
dures we have seen in previous chapters for manipulating lists. The following 
subsection introduces some imperative control structures. 

9.4.1 ListMutators 

All thè procedures for changing thè value of a list in Section 5.4.3 actually do 
not change any values; instead they construct new lists. When our goal is only 
to change some elements in an existing list, this wastes memory constructing a 
new list and may require more running time than a procedure that modifies thè 
input list instead. Here, we revisit some of thè procedures from Section 5.4.3, 
but instead of producing new lists with thè desired property these procedures 
modify thè input list. 


Example 9.2: Mapping 


The list-map procedure (from Example 5.4) produces a new list that is thè result 
of applying thè same procedure to every element in thè input list. 

(define {list-map f p) 

(if {nuli? p) nidi {cons (/ {car p)) {list-map f ( cdr p))))) 
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Whereas thè functional list-map procedure uses cons to build up thè output list, 
thè imperative mlist-map! procedure uses set-car! to mutate thè input list’s ele- 
ments: 

(define {mlist-map! f p) 

(if ( nuli ? p) {void) 

(begin {set-mcar! p (/ ( mcar p ) ) ) 

{mlist-map! f {mcdr p))))) 

The base case uses {void) to evaluate to no value. Unlike list-map which evalu- 
ates to a List, mlist-map! is evaluated for its side effects and produces no output. 

Assuming thè procedure passed as / has Constant running time, thè running 
time of thè mlist-map! procedure is in 0(n) where n is thè number of elements 
in thè input list. There will be n recursive applications of mlist-map! since each 
one passes in a list one element shorter than thè input list, and each application 
requires Constant time. This is asymptotically thè same as thè list-map proce- 
dure, but we would expect thè actual running time to be faster since there is no 
need to construct a new list. 

The memory consumed is asymptotically different. The list-map procedure al- 
locates n new cons cells, so it requires memory in 0(n) where n is thè number of 
elements in thè input list. The mlist-map! procedure is tail recursive (so no staclc 
needs to be maintained) and does not allocate any new cons cells, so it requires 
Constant memory. 


Example 9.3: Filtering 


The list-fllter procedure takes as inputs a test procedure and a list and outputs 
a list containing thè elements of thè input list for which applying thè test proce- 
dure evaluates to a true value. In Example 5.5, we defined list-fllter as: 

(define {list-fllter test p) 

(if {nuli? p) nuli 

(if {test {car p)) {cons {car p) {list-fllter test {cdr p))) 

{list-fllter test {cdr p))))) 

An imperative version of list-fllter removes thè unsatisfying elements from a 
mutable list. We define mlist-fllter! using set-mcdr! to skip over elements that 
should not be included in thè filtered list: 

(define {mlist-fllter! test p) 

(if {nuli? p) nuli 

(begin {set-mcdr! p {mlist-fllter! test {mcdr p))) 

(if {test {mcar p)) p {mcdr p))))) 

Assuming thè test procedure has Constant running time, thè running time of thè 
mlist-fllter! procedure is linear in thè length of thè input list. As with mlist-map!, 
thè space used by mlist-fllter! is Constant, which is better than thè 0(n) space 
used by list-fllter. 

Unlike mlist-map!, mlist-fllter! outputs a value. This is needed when thè first 
element is not in thè list. Consider this example: 

> (define a {mlist 12 3 14)) 

> {mlist-fllter! (lambda (x) (> x 1)) a) 

{2 3 4} 
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> a 

{1 2 3 4} 

The value of a stili includes thè initial 1 . There is no way for thè mlist-fllter! pro- 
cedure to remove thè fìrst element of thè list: thè set-mcar! and set-mcdr! proce- 
dures only enable us to change what thè mutable pair’s components contain. 

To avoid this, mlist-fllter! should be used with set! to assign thè variable to thè 
resulting mutable list: 

(set! a ( mlist-fllter ! (lambda (x) (> x 1)) a)) 


Example 9.4: Append 


The list-append procedure takes as input two lists and produces a list consisting 
of thè elements of thè fìrst list followed by thè elements of thè second list. An 
imperative version of this procedure instead mutates thè fìrst list to contain thè 
elements of both lists. 

(define ( mlist-append ! p q ) 

(if ( nuli ? p) ( error "Cannot append to an empty list") 

(if ( nuli ? ( mcdr p)) ( set-mcdr ! p q) 

( mlist-append ! ( mcdr p) q)))) 

The mlist-append! procedure produces an error when thè fìrst input is nuli — 
this is necessary since if thè input is nuli there is no pair to modify. 3 

Like list-append, thè running time of thè mlist-append! procedure is in 0(n) 
where n is thè number of elements in thè fìrst input list. The list-append pro- 
cedure copies thè fìrst input list, so its memory use is in 0(n) where n is thè 
number of elements in thè fìrst input list. The memory use of mlist-append! is 
Constant: it does not create any new cons cells to append thè lists. 


Aliasing. Adding mutation makes it possible to defìne many procedures more 
efficiently and compactly, but introduces many new potential pitfalls in produc- 
ing reliable programs. Since our evaluation model now depends on thè environ- 
ment in which an expression is evaluated, it becomes much harder to reason 
about code by itself. 

aliasing One challenge introduced by mutation is aliasing. There may be different ways 
to refer to thè same object. This was true before mutation also, but didn’t mat- 
ter since thè value of an object never changed. Once object values can change, 
however, aliasing can lead to surprising behaviors. 

For example, 

> (define mi ( mlist 1 2 3)) 

> (define m2 ( mlist 4 5 6)) 

> ( mlist-append ! mi m2) 

> (set! mi ( mlist-fllter ! (lambda (e/) {— ( modulo el 2) 0)) mi)) 


3 The mappend! library procedure in DrRacket takes a different approach: when thè fìrst input 
is nuli it produces thè value of thè second list as output in this case. This has unexpected behavior 
when an expression like [appendi a b) is evaluated where thè value of a is nidi since thè value of a is 
not modifìed. 
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The value of m2 was defined as {4 5 6}, and no expressions since then explicitly 
modified m2. But, thè value of m2 has stili changed! It changed because after 
evaluating ( mlist-append ! mi m2) thè mi object shares cells with m2. Thus, 
when thè mlist-fllter! application changes thè value of mi, it also changes thè 
value of m2 to {4 6}. 

The built-in procedure eq? takes as input any two objects and outputs a Boolean. 
The result is true if and only if thè inputs are thè same object. For example, (eq? 
3 3) evaluates to true but (eq? (mcons 1 2) (mcons 1 2)) evaluates to false. Even 
though thè input pairs have thè same value, they are different objects — mutating 
one of thè pairs does not effect thè value of thè other pair. 

For thè earlier mlist-append! example, (eq? mi m2) evaluates to false since mi 
and m2 do not refer to thè same object. But, (eq? (mcdr mi) m2) evaluates to 
true since thè second celi of mi points to thè same object as m2. Evaluating 
(set-mcar! m2 3) changes thè value of both mi and m2 since thè modified celi is 
common to both structures. 


Exercise 9.6. Define an imperative-style procedure, mlist-inc! that takes as in- 
put a MutableList of Numbers and modifies thè list by adding one to thè value 
of each element in thè list. 


Exercise 9.7. [*] Define a procedure mlist-truncate! that takes as input a Mu- 
tableList and modifies thè list by removing thè last element in thè list. Specify 
carefully thè requirements for thè input list to your procedure. 

Exercise 9.8. [*] Define a procedure mlist-make-circular! that takes as input a 
MutableList and modifies thè list to be a circular list containing all thè elements 
in thè originai list. For example, (mlist-make-circular! (mlist 3)) should produce 
thè same structure as thè circular pair shown in Figure 9.5. 

Exercise 9.9. [*] Define an imperative-style procedure, mlist-reverse! , that re- 
verses thè elements of a list. Is it possible to implement a mlist-reverse! pro- 
cedure that is asymptotically faster than thè list-reverse procedure from Exam- 
ple 5.4? 


Ifyou steal property, 
you must report its 
fair market value in 
your incoine in thè 
yearyou steal it 
unless in thè same 
year, you return it to 
its rightful owner. 
Your Federai Incorile 
Tax, IRS Publication 
17 , 2009 . 


Exercise 9.10. [**" Define a procedure mlist- aliases? that takes as input two 
mutable lists and outputs true if and only if there are any mcons cells shared 
between thè two lists. 

9.4.2 Imperative Control Structures 

The imperative style of programming makes progress by using assignments to 
manipulate state. In many cases, solving a problem requires repeated opera- 
tions. With functional programming, this is done using recursive definitions. 
We make progress towards a base case by passing in different values for thè 
operands with each recursive application. With imperative programming, we 
can make progress by changing state repeatedly without needing to pass in dif- 
ferent operands. 
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while loop A common control structure in imperative programming is a while loop. A while 
loop has a test condition and a body. The test condition is a predicate. If it 
evaluates to true, thè while loop body is executed. Then, thè test condition is 
evaluated again. The while loop continues to execute until thè test condition 
evaluates to false. 

We can defìne while as a procedure that takes as input two procedures, a test 
procedure and a body procedure, each of which take no parameters. Even though 
thè test and body procedures take no parameters, they need to be procedures in- 
stead of expressions, since every iteration of thè loop should re-evaluate thè test 
and body expressions of thè passed procedures. 

(define [while test body ) 

(if [test) 

(begin [body) [while test body)) 

[void))) ; no result value 

We can use thè while procedure to implement Fibonacci similarly to thè fast- 
flbo procedure: 

(define [fibo-while n) 

(let ((« 1) [b 1)) 

[while (lambda () (> n 2)) 

(lambda () (set! b [+ a b)) 

(set! a[— b a)) 

(set! n (- n 1)))) 
b) ) 

The final value of b is thè result of thè fibo- while procedure. In each iteration, thè 
body procedure is applied, updating thè values of a and b to thè next Fibonacci 
numbers. 

The value assigned to a is computed as [— b a) instead of b. The reason for this 
is thè previous assignment expression has already changed thè value of b, by 
adding a to it. Since thè next value of a should be thè old value of b, we can 
find thè necessary value by subtracting a. The fact that thè value of a variable 
can change depending on when it is used often makes imperative programming 
trickier than functional programming. 

An alternative approach, which would save thè need to do subtraction, is to sto re 
thè old value in a temporary value: 

(lambda () 

(let [[oldb b)) 

(set! b [+ a b)) 

(set! a oldb) 

(set! n[— n 1)))) 

Many programming languages designed to support imperative programming 
provide control constructs similar to thè while procedure defined above. For ex- 
ampie, here is a version of thè procedure in thè Python programming language: 

deifibonacci [ ri) : 
a- 1 
b= 1 
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while n > 2 : 

a,b-b,a+b 
n-n — 1 

return b 


We will use Python starting in Chapter 11, although you can probably guess what 
most of this procedure means without knowing Python. 

The most interesting statement is thè doublé assignment: a, b = b, a + b. This 
assigns thè new value of a to thè old value of b, and thè newvalue of b to thè sum 
of thè old values of a and b. Without thè doublé assignment operator, it would 
be necessary to store thè old value of b in a new variable so it can be assigned to 
a after updating b to thè newvalue. 


Exercise 9.11. Defìne thè mlist-map! example from thè previous section using 
while. 

Exercise 9.12. Another common imperative programming structure is a repeat- 
until loop. Define a repeat-until procedure that takes two inputs, a body proce- 
dure and a test procedure. The procedure should evaluate thè body procedure 
repeatedly, until thè test procedure evaluates to a true value. For example, using 
repeat-until we could defin efactorial as: 

(define ( factorial n) 

(let [[fact 1)) 

(. repeat-until 

(lambda () (set! fact (* faci n) ) (set! n (- ni))) 

(lambda () (< n 1))) 
fact)) 


Exercise 9.13. [**] Improve thè efficiency of thè indexing procedures from Sec- 
tion 8.2.3 by using mutation. Start by defining a mutable binary tree abstraction, 
and then use this and thè MutableList data type to implement an imperative- 
style insert-into- index! procedure that mutates thè input index by adding a new 
word-position pair to it. Then, define an efficient merge-index! procedure that 
takes two mutable indexes as its inputs and modifies thè first index to incor- 
porate all word occurrences in thè second index. Analyze thè impact of your 
changes on thè running time of indexing a collection of documents. 

9.5 Summary 

Adding thè ability to change thè value associated with a name complicates our 
evaluation rules, but enables simpler and more efficient Solutions to many prob- 
lems. Mutation allows us to efficiently manipulate larger data structures since it 
is not necessary to copy thè data structure to make changes to it. 

Once we add assignment to our language, thè order in which things happen 
affects thè value of some expressions. Instead of evaluating expressions using 
substitution, we now need to always evaluate an expression in a particular exe- 
cution environment. 
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9.5. Summary 


The problem with mutation is that it makes it much tougher to reason about 
thè meaning of an expression. In thè next chapter, we introduce a new kind of 
abstraction that packages procedures with thè state they manipulate. This helps 
manage some of thè complexity resulting from mutation by limiting thè places 
where data may be accessed and modified. 


10 

Objects 


It was amazing to me, and it is stili amazing, that people could not ìmagine what thè 
psychological difference would be to bave an interactive terminal. You can talk about it on a 
blackboard. until you are blue in thè face, and people would say, "Oh, yes, but why do you need 
that?’’. . . We used to try to think ofall these analogies, like describing it in terms ofthe 
difference between mailing a letter to your mother and getting on thè telephone. To this day I 
can stili remember people only realizing when they sawa reai demo, say, "Hey, it talks back. 

Wow! You just type that and you got an answer." 
Fernando Corbató (who worked on Whirlwind in thè 1950s) 
Charles Babbage Institute interview, 1989 

So far, we have seen two main approaches for solving problems: 

Functional programming 

Break a problem into a group of simpler procedures that can be composed 
to solve thè problem (introduced in Chapter 4). 

Data-centric programming 

Model thè data thè problem involves, and develop procedures to manip- 
olate that data (introduced in Chapter 5, and extended to imperative pro- 
gramming with mutation in thè previous chapter) . 

All computational problems involve both data and procedures. All procedures 
act on some form of data; without data they can have no meaningful inputs and 
outputs. Any data-focused design must involve some procedures to perform 
computations using that data. 

This chapter introduces a new problem-solving approach known as object-ori- 
ented programming. By packaging procedures and data together, object-orien- 
ted programming overcomes a weakness of both previous approaches: thè data 
and thè procedures that manipulate it are separate. 

Unlike many programming languages, Scheme does not provide special built- 
in support for objects. 1 We build an object System ourselves, taking advantage 
of thè stateful evaluation rules. By building an object System from simple com- 
ponents, we provide a clearer and deeper understanding of how object Systems 
work. In Chapter il, we see how Python provides language support for object- 
oriented programming. 

The next section introduces techniques for programming with objects that com- 
bine state with procedures that manipulate that state. Section 10.2 describes 

^his refers to thè standard Scheme language, not thè many extended Scheme languages pro- 
vided by DrRacket. The MzScheme language does provide additional constructs for supporting ob- 
jects, but we do not cover them in this hook. 


object-oriented 

programming 
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10.1. Packaging Procedures and State 


inheritance inheritance, a powerful technique for programming with objects by implement- 
ing new objects that add or modify thè behaviors of previously implemented 
objects. Section 10.3 provides some historical background on thè development 
of object-oriented programming. 

10.1 Packaging Procedures and State 

Recali our counter from Example 9.1: 

(define ( update-counter !) (set! counter (+ counter 1)) counter) 

Every time an application of update-counter! is evaluated, we expect to obtain a 
value one larger than thè previous application. This only works, however, if there 
are no other evaluations that modify thè counter variable. Hence, we can only 
have one counter: there is only one counter place in thè global environment. 
If we want to have a second counter, we would need to define a new variable 
(such as counter2, and implement a new procedure, update-counter2! , that is 
identical to update-counter!, but manipulates counter2 instead. For each new 
counter, we would need a new variable and a new procedure. 

10.1.1 Encapsulation 

It would be more useful to package thè counter variable with thè procedure that 
manipulates it. Then we could create as many counters as we want, each with 
its own counter variable to manipulate. 

The Stateful Application Rule (from Section 9.2.2) suggests a way to do this: eval- 
uating an application creates a new environment, so a counter variable defined 
an thè application environment is only visible through body of thè created pro- 
cedure. 

The make-counter procedure creates a counter object that packages thè count 
variable with thè procedure that increases its value: 

(define ( make-counter ) 

((lambda {count) 

(lambda () (set! count (+ 1 count)) count)) 

0 )) 

Each application of make-counter produces a new object that is a procedure 
with its own associated count variable. Protecting state so it can only be manip- 
encapsulation ulated in controlled ways is known as encapsulation. 

The count place is encapsulated within thè counter object. Whereas thè previ- 
ous counter used thè global environment to store thè counter in a way that could 
be manipulated by other expressions, this version encapsulates thè counter vari- 
able so thè only way to manipulate thè counter value is through thè counter ob- 
ject. 

An equivalent make-counter definition uses a let expression to make thè initial- 
ization of thè count variable clearer: 

(define {make-counter) 

(let {{count 0)) 

(lambda () (set! count (+ 1 count)) count))) 
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Figure 10.1 depicts thè environment after creating two counter objects and ap- 
plying one of them. 


Global Environment 



body: 

(begin (set! count (+ 1 count ) 
v count)) 


body: 

(begin (set! count (+ 1 count) 
count)) 


Figure 10.1. Environment produced by evaluating: 
(define counterl ( make-counter )) 

(define counter2 ( make-counter )) 

( counterl ) 


10.1.2 Messages 

The object produced by make-counter is limited to only one behavior: every 
time it is applied thè associated count variable is increased by one and thè new 
value is output. To produce more useful objects, we need a way to combine state 
with multiple behaviors. 

For example, we might want a counter that can also return thè current count 
and reset thè count. We do this by adding a message parameter to thè procedure 
produced by make-counter: 

(define ( make-counter ) 

(let {{count 0)) 

(lambda {message) 

(if {eq? message ’get-count) count 

(if {eq? message 'reset!) (set! count 0) 

(if {eq? message ’next!) (set! count (+ 1 count)) 

{error "Unrecognized message"))))))) 

Like thè earlier make-counter, this procedure produces a procedure with an en- 
vironment containing a frame with a place named count. The produced proce- 
dure takes a message parameter and selects different behavior depending on thè 
input message. 

The message parameter is a Symbol. A Symbol is a sequence of characters pre- 
ceded by a quote character such as ’next!. Two Symbols are equal, as determined 
by thè eq? procedure, if their sequences of characters are identical. The running 
time of thè eq? procedure on Symbol type inputs is Constant; it does not increase 
with thè length of thè symbols since thè symbols can be represented internally 
as small numbers and compared quickly using number equality. This makes 
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symbols a more effìcient way of selecting object behaviors than Strings, and a 
more memorable way to select behaviors than using Numbers. 

Here are some sample interactions using thè counter object: 

> (define counter ( make-counter )) 

> ( counter ’next!) 

> ( counter ’get-count) 

1 

> ( counter ’previous!) 

G Unrecognized message 

Conditional expressions. For objects with many behaviors, thè nested if ex- 
pressions can get quite cumbersome. Scheme provides a compact conditional 
expression for combining many if expressions into one smaller expression: 

Expression ::=> CondExpression 

Con d Expression: : => (cond CondClauseList ) 

Con d Cla useList : : => CondClause CondClauseList 
CondClauseList ::=> e 

CondClause ::=> (Expression praiicate Expression C0nsequent ) 

The evaluation rule for a conditional expression can be defìned as a transforma- 
tion into an if expression: 

Evaluation Rule 9: Conditional. The conditional expression (cond) 
has no value. All other conditional expressions are of thè form (cond 
(Expressionpi Expression cl ) Resti where Rest is a list of conditional 
clauses. The value of such a conditional expression is thè value of thè 
if expression: 

(if Expression p i Expression^ (cond Rest)) 

This evaluation rule is recursive since thè transformed expression stili includes a 
conditional expression, but uses thè empty conditional with no value as its base 
case. 

The conditional expression can be used to defìne make-counter more clearly 
than thè nested if expressions: 

(define ( make-counter ) 

(let ((counto)) 

(lambda (message) 

(cond ((eq? message ’get-count) count) 

((eq? message 'reset!) (set! conni 0)) 

((eq? message ’next!) (set! count (+ 1 count))) 

(true (error "Unrecognized message")))))) 

For linguistic convenience, Scheme provides a special syntax else for use in con- 
ditional expressions. When used as thè predicate in thè last conditional clause 
it means thè same thing as true. So, we could write thè last clause equivalently 
as (else (error "Unrecognized message")). 

Sending messages. A more naturai way to interact with objects is to defìne 
a generic procedure that takes an object and a message as its parameters, and 
send thè message to thè object. 
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The ask procedure is a simple procedure that does this: 

(define {ask object message ) ( object message )) 

It applies thè object input to thè message input. So, ( ask counter ’next!) is equiv- 
alent to ( counter ’next!), but looks more like passing a message to an object than 
applying a procedure. Later, we will develop more complex versions of thè ask 
procedure to provide a more powerful object model. 

Messages with parameters. Sometimes it is useful to have behaviors that take 
additional parameters. For example, we may want to support a message adjust! 
that increases thè counter value by an input value. 

To support such behaviors, we generalize thè behaviors so that thè result of ap- 
plying thè message dispatching procedure is itself a procedure. The procedures 
for reset!, next!, and get-count take no parameters; thè procedure for adjust! 
takes one parameter. 

(define ( make-adj uslable-counler) 

(let {{count 0)) 

(lambda ( message ) 

(cond {{eq? message ’get-count) (lambda () count)) 

{{eq? message 'reset!) (lambda () (set! count 0))) 

{{eq? message ’next!) (lambda () (set! count (+ 1 count)))) 

{{eq? message ’adjust!) 

(lambda {vai) (set! count (+ count vai)))) 

(else {error "Unrecognized message")))))) 

We also need to also change thè ask procedure to pass in thè extra arguments. 

So far, all thè procedures we have defined take a fìxed number of operands. To 
allow ask to work for procedures that take a variable number of arguments, we 
use a special definition construct: 

Definition ::=> (define {Name Parameters . Name Rest ) Expression) 

The name following thè dot is bound to all thè remaining operands combined 
into a list. This means thè defined procedure can be applied to n or more operands 
where n is thè number of names in Parameters. If there are only n operand ex- 
pressions, thè value bound to Name Rest is nuli. If there are n + k operand expres- 
sions, thè value bound to Name Rest is a list containing thè values of thè last k 
operand expressions. 

To apply thè procedure we use thè built-in apply procedure which takes two 
inputs, a Procedure and a List. It applies thè procedure to thè values in thè list, 
extracting them from thè list as each operand in order. 

(define {ask object message . args) 

{apply {object message) args)) 

We can use thè new ask procedure with two or more parameters to invoke meth- 
ods with any number of arguments (e.g., > {ask counter ’adjust! 5)). 

10.1.3 Object Terminology 

An object is an entity that packages state and procedures. object 

The state variables that are part of an object are called instance variables. The instance variables 
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instance variables are stored in places that are part of thè application environ- 
ment for thè object. This means they are encapsulated with thè object and can 
only be accessed through thè object. An object produced by ( make-counter ) de- 
fines a single instance variable, count. 

methods The procedures that are part of an object are called methods. Methods may pro- 
vide information about thè state of an object (we cali these observers ) or modify 
thè state of an object (we cali these mutators). An object produced by ( make- 
counter •) provides three methods: reset! (a mutator), next! (a mutator), and gel- 
ee) uni (an observer). 

invoke An object is manipulated using thè object’s methods. We invoke a method on an 
object by sending thè object a message. This is analogous to applying a proce- 
dure. 

class A class is a kind of object. Classes are similar to data types. They define a set of 
possible values and operations (methods in thè object terminology) for manip- 
ulating those values. We also need procedures for creating new objects, such as 
constructors thè make-counter procedure above. We cali these constructors. By convention, 
we cali thè constructor for a class make-<class> where <class> is thè name of 
thè class. Hence, an instance of thè counter class is thè result produced when 
thè make-counter procedure is applied. 


Exercise 10.1. Modify thè make-counter definition to add a previous! method 
that decrements thè counter value by one. 

Exercise 10.2. [*] Define a variable-counter object that provides these meth- 
ods: 

make-variable-counter. Number — > VariableCounter 

Creates a variable-counter object with an initial counter value of 0 and an 
initial increment value given by thè parameter. 

set-incrementi: Number — > Void 

Sets thè increment amount for this counter to thè input value. 

next 1: Void — > Void 

Adds thè increment amount to thè value of thè counter. 

get-count : Void — > Number 

Outputs thè current value of thè counter. 

Here are some sample interactions using a variable-counter object: 

> (define vcounter ( make-variable-counter 1)) 

> ( ask vcounter ’next!) 

> ( ask vcounter ’set-increment! 2) 

> ( ask vcounter ’next!) 

> ( ask vcounter ’get-count) 

3 

10.2 Inheritance 

Objects are particularly well-suited to programs that involve modeling reai or 
imaginary worlds such as graphical user interfaces (modeling Windows, fìles, 
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and folders on a desktop), simulations (modeling physical objects in thè reai 
world and their interactions), and games (modeling creatures and things in an 
imagined world) . 

Objects in thè reai world (or most simulated worlds) are complex. Suppose we 
are implementing a game that simulates a typical university. It might include 
many different kinds of objects including places (which are stationary and may 
contain other objects), things, and people. There are many different kinds of 
people, such as students and professors. All objects in our game have a name 
and a location; some objects also have methods for talking and moving. We 
could define classes independently for all of thè object types, but this would 
involve a lot of duplicate effort. It would also make it hard to add a new behavior 
to all of thè objects in thè game without modifying many different procedures. 

The solution is to define more specialized kinds of objects using thè definitions 
of other objects. For example, a student is a kind of persoti. A student has all 
thè behaviors of a normal persoti, as well as some behaviors particular to a stu- 
dent such as choosing a major and graduating. To implement a student class, 
we want to reuse methods from thè person class without needing to duplicate 
them in thè student implementation. We cali thè more specialized class (in this 
case thè student class) thè subclass and say student is a subclass of person. The subclass 
reused class is known as thè superclass, so person is thè superclass of student. A superclass 
class can have many subclasses but only one superclass. 2 

Figure 10.2 illustrates some inheritance relationships for a university simulator. 

The arrows point from subclasses to their superclass. A class may be both a 
subclass to another class, and a superclass to a different class. For example, 
person is a subclass of movable- object, but a superclass of student and professor. 

sim-object 

Z 

movable-object place 

? Z 

thing person 

student professor 

Figure 10.2. Inheritance hierarchy. 


Our goal is to be able to reuse superclass methods in subclasses. When a method 
is invoked in a subclass, if thè subclass does not provide a definition of thè 
method, then thè definition of thè method in thè superclass is used. This can 
continue up thè superclass chain. For instance, student is a subclass of person, 
which is a subclass of movable-object, which is a subclass of sim-object (simula- 
tion object), which is thè superclass of all classes in thè simulator. 

2 Some object Systems (such as thè one provided by thè C++ programming language) allow a class 
to have more than one superclass. This can be confusing, though. If a class has two superclasses and 
both define methods with thè same name, it may be ambiguous which of thè methods is used when 
it is invoked on an object of thè subclass. In our object System, a class may have only one superclass. 
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Hence, if thè sim-object class defines a get-name method, when thè get-name 
method is invoked on a student object, thè implementation of get-name in thè 
sim-object class will be used (as long as neither person nor movable-object de- 
fines its own get-name method). 

When one class implementation uses thè methods from another class we say thè 
inherits subclass inherits from thè superclass. Inheritance is a powerful way to obtain 
many different objects with a small amount of code. 

10.2.1 Implementing Subclasses 

To implement inheritance we change class definitions so that if a requested 
method is not defined by thè subclass, thè method defined by its superclass will 
be used. 

The make-sub-object procedure does this. It takes two inputs, a superclass ob- 
ject and thè object dispatching procedure of thè subclass, and produces an in- 
stance of thè subclass which is a procedure that takes a message as input and 
outputs thè method corresponding to that message. If thè method is defined by 
thè subclass, thè result will be thè subclass method. If thè method is not defined 
by thè subclass, it will be thè superclass method. 

(define ( make-sub-object super subproc) 

(lambda ( message ) 

(let (( method ( subproc message))) 

(if method method ( super message))))) 

When an object produced by ( make-sub-object obj proc) is applied to a mes- 
sage, it first applies thè subclass dispatch procedure to thè message to find an 
appropriate method if one is defined. If no method is defined by thè subclass 
implementation, it evaluates to ( super message), thè method associated with 
thè message in thè superclass. 

References to self. It is useful to add an extra parameter to all methods so thè 
object on which thè method was invoked is visible. Otherwise, thè object will 
lose its special behaviors as it is moves up thè superclasses. We cali this thè self 
object (in some languages it is called thè this object instead). To support this, 
we modify thè ask procedure to pass in thè object parameter to thè method: 

(define {ask object message . args) 

(i apply ( object message) object args)) 

All methods now take thè self object as their first parameter, and may take addi- 
tional parameters. So, thè counter constructor is defined as: 

(define ( make-counter ) 

(let {{counto)) 

(lambda {message) 

(cond 

(( eq ? message ’get-count) (lambda {self) count)) 

{{eq? message 'reset!) (lambda {self) (set! count 0))) 

{{eq? message ’next!) (lambda {self) (set! count (+ 1 count)))) 

(else {error "Unrecognized message")))))) 

Subclassing counter. Since subclass objects cannot see thè instance variables 
of their superclass objects directly, if we want to provide a versatile counter class 
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we need to also provide a set-count! method for setting thè value of thè counter 
to an arbitrary value. For reasons that will become clear later, we should use set- 
count! everywhere thè value of thè count variable is changed instead of setting 
it directly: 

(define ( make-counter ) 

(let ((count 0)) 

(lambda (message) 

(cond 

((eq? message ’get-count) (lambda (self) count)) 

((eq? message ’set-count!) (lambda (self vai) (set! count vai))) 

((eq? message 'reset!) (lambda (self) (ask self ’set-count! 0))) 

((eq? message ’next!) 

(lambda (self) (ask self ’set-count! (+ 1 (ask self ’current))))) 

(else (error "Unrecognized message")))))) 

Previously, we defìned make-adjustable-counter by repeating all thè code from 
make-counter and adding an adjust! method. With inheritance, we can defìne 
make-adjustable-counter as a subclass of make-counter without repeating any 
code: 

(define (make-adjustable-counter) 

(make-sub-object 

(make-counter) 

(lambda (message) 

(cond 

((eq? message ’adjust!) 

(lambda (self vai) 

(ask self ’set-count! (+ (ask self ’get-count) vai)))) 

(élse false))))) 

We use make-sub-object to create an object that inherits thè behaviors from one 
class, and extends those behaviors by defining new methods in thè subclass im- 
plementation. 

The new adjust! method takes one Number parameter (in addition to thè self 
object that is passed to every method) and adds that number to thè current 
counter value. Itcannotuse (set! count (+ count vai)) directly, though, sincethe 
count variable is defìned in thè application environment of its superclass object 
and is not visible within adjustable- counter. Hence, it accesses thè counter us- 
ing thè set-count! and get-count methods provided by thè superclass. 

Suppose we create an adjustable-counter object: 

(define acounter (make-adjustable-counter)) 

Considerwhathappens when (ask acounter ’adjust! 3) is evaluated. The acounter 
object is thè result of thè application of make-sub-object which is thè procedure, 

(lambda (message) 

(let ((method (subproc message))) 

(if method method (super message))))) 

where super is thè counter object resulting from evaluating (make-counter) and 
subproc is thè procedure created by thè lambda expression in make- adjustable- 
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counter. The body of ask evaluates ( object messagè) to find thè method associ- 
ated with thè input message, in this case ’adjust!. The acounter object takes thè 
message input and evaluates thè let expression: 

(let (( method ( subproc message))) . . .) 

The result of applying subproc to message is thè adjust! procedure defined by 
make-adjustable-counter : 

(lambda ( self vai) 

(i ask self ’set-count! (+ ( ask self ’get-count) vai))) 

Since this is not false, thè predicate of thè if expression is non-false and thè value 
of thè consequent expression, method, is thè result of thè procedure application. 
The ask procedure uses apply to apply this procedure to thè object and args 
parameters. The object is thè acounter object, and thè args is thè list of thè extra 
parameters, in this case (3) . 

Thus, thè adjust! method is applied to thè acounter object and 3. The body of 
thè adjust! method uses ask to invoke thè set-count! method on thè self object. 
As with thè first invocation, thè body of ask evaluates ( object message) to find 
thè method. In this case, thè subclass implementation provides no set-count! 
method so thè result of ( subproc message) in thè application of thè subclass ob- 
ject is false. Hence, thè alternate expression is evaluated: ( super message). This 
evaluates to thè method associated with thè set-count! message in thè super- 
class. The ask body will apply this method to thè self object, setting thè value of 
thè counter to thè new value. 

We can define new classes by defining subclasses of previously defined classes. 
For example, reversible-counter inherits from adjustable- counter. 

(defìne ( make- reversible-co un ter) 

( make-subobject 
( make-adjustable-counter ) 

(lambda ( message ) 

(cond 

[[eq? message ’previous!) (lambda [self) [ask self ’adjust! -1))) 
(else/aZse))))) 

The reversible-counter object defines thè previous! method which provides a 
new behavior. If thè message to a adjustable-counter object is not previous!, 
thè method from its superclass, adjustable-counter is used. Within thè previous! 
method we use ask to invoke thè adjust! method on thè self object. Since thè 
subclass implementation does not provide an adjust! method, this results in thè 
superclass method being applied. 

10.2.2 Overriding Methods 

In addition to adding new methods, subclasses can replace thè definitions of 
methods defined in thè superclass. When a subclass replaces a method defined 
overrìdes by its superclass, then thè subclass method overrides thè superclass method. 
When thè method is invoked on a subclass object, thè new method will be used. 

For example, we can define a subclass of reversible-counter that is not allowed 
to have negative counter values. If thè counter would reach a negative number, 
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instead of setting thè counter to thè new value, it produces an errar message and 
maintains thè counter at zero. We do this by overriding thè set-count! method, 
replacing thè superclass implementation of thè method with a new implemen- 
tation. 

(define ( make-positive-counter ) 

( make-subobject 
( make-reversible-counter ) 

(lambda ( message ) 

(cond 

((eq? message ’set-count!) 

(lambda ( self vai) (if (< vai 0) ( error "Negative count") 

...))) 

(else false))))) 

What should go in place of thè . . .? When thè value to set thè count to is not 
negative, what should happen is thè count is set as it would be by thè super- 
class set-count! method. In thè positive-counter code though, there is no way 
to access thè count variable since it is in thè superclass procedure’s application 
environment. There is also no way to invoke thè superclass’ set-count! method 
since it has been overridden by positive-counter. 

The solution is to provide a way for thè subclass object to obtain its superclass 
object. We can do this by adding a get-super method to thè object produced by 
malce-sub-object: 

(define ( make-sub-object super subproc) 

(lambda ( message ) 

(if (eq? message ’get-super) 

(lambda (self) super) 

(let (( method (subproc message))) 

(if method method (super message)))))) 

Thus, when an object produced by make-sub-object is passed thè get-super mes- 
sage it returns a method that produces thè super object. The rest of thè proce- 
dure is thè same as before, so for every other message it behaves like thè earlier 
make-sub-object procedure. 

With thè get-super method we can define thè set-count! method for positive- 
counter, replacing thè . . . with: 

(ask (ask self ’get-super) ’set-count! vai)) 

Figure 10.3 shows thè subclasses that inherit from counter and thè methods they 
define or override. 

Consider these sample interactions with a positive-counter object: 

> (define poscount (make-positive-counter)) 

> (ask poscount ’next!) 

> (ask poscount ’previous!) 

> (ask poscount ’previous!) 

Q Negative count 

> (ask poscount ’get-count) 

0 
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Figure 10.3. Counter class hierarchy. 


For thè fìrst ask application, thè next! method is invoked on a positive-counter 
object. Since thè positive-counter class does not define a next! method, thè mes- 
sage is sent to thè superclass, reversible-counter. The reversible-counter imple- 
mentation also does not define a next! method, so thè message is passed up to its 
superclass, adjustable-counter. This class also does not define a next! method, 
so thè message is passed up to its superclass, counter. The counter class defines 
a next! method, so that method is used. 

For thè next ask, thè previous! method is invoked. Since thè positive-counter 
class does not define a previous! method, thè message is sent to thè superclass. 
The superclass, reversible-counter, defines a previous! method. Its implemen- 
tation involves an invocation of thè adjust! method: ( ask self ’adjust! -1). This 
invocation is done on thè self object, which is an instance of thè positive-counter 
class. Hence, thè adjust! method is found from thè positive-counter class imple- 
mentation. This is thè method that overrides thè adjust! method defined by thè 
adjustable-counter class. Hence, thè second invocation of previous! produces 
thè “Negative count” error and does not adjust thè count to -1 . 

The property this object System has where thè method invoked depends on 
dynamic dispatch thè object is known as dynamic dispatch. The method used for an invocation 
depends on thè self object. In this case, for example, it means that when we 
inspect thè implementation of thè previous! method in thè reversible-counter 
class by itself it is not possible to determine what procedure will be applied for 
thè method invocation, ( ask self ’adjust! -1). It depends on thè actual self ob- 
ject: if it is a positive-counter object, thè adjust! method defined by positive- 
counter is used; if it is a reversible-counter object, thè adjust! method defined by 
adjustable-counter class (thè superclass of reversible-counter) is used. 

Dynamic dispatch provides for a great deal of expressiveness. It enables us to 
use thè same code to produce many different behaviors by overriding methods 
in subclasses. This is very useful, but also very dangerous — it makes it impos- 
sible to reason about what a given procedure does, without knowing about all 
possible subclasses. For example, we cannot make any claims about what thè 
previous! method in reversible-counter actually does without knowing what thè 
adjust! method does in all subclasses of reversible-counter. 
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The value of encapsulation and inheritance increases as programs get more com- 
plex. Programming with objects allows a programmer to manage complexity by 
hiding thè details of how objects are implemented from what those objects rep- 
resent and do. 


Exercise 10.3. Define a countdown class that simulates a rocket launch count- 
down: it starts at some initial value, and counts down to zero, at which point thè 
rocket is launched. Can you implement countdown as a subclass of counter ? 

Exercise 10.4. Define thè variable-counter object from Exercise 10.2 as a sub- 
class of counter. 

Exercise 10.5. Define a new subclass of parameterizable-counter where thè in- 
crement for each next! method application is a parameter to thè constructor 
procedure. For example, ( make-parameterizable-counter 0.1) would produce 
a counter object whose counter has value 0.1 after one invocation of thè next! 
method. 


10.3 Object-Oriented Programming 

Object-Oriented programming is a style of programming where programs are 
broken down into objects that can be combined to solve a problem or model a 
simulated world. The notion of designing programs around object manipula- 
tions goes back at least to Ada (see thè quote at thè end if Chapter 6), but started 
in earnest in thè early 1960s. 

During World War II, thè US Navy began to consider thè possibility of building a 
airplane simulator for training pilots and aiding aircraft designers. At thè time, 
pilots trained in mechanical simulators that were custom designed for partic- 
ular airplanes. The Navy wanted a simulator that could be used for multiple 
airplanes and could accurately model thè aerodynamics of different airplanes. 

Project Whirlwind was started at MIT to build thè simulator. The initial plans 
called for an analog computer which would need to be manually reconfigured 
to change thè aerodynamics model to a different airplane. Jay Forrester learned 
about emerging projects to build digitai computers, including ENIAC which be- 
came operational in 1946, and realized that building a programmable digitai 
computer would enable a much more flexible and powerful simulator, as well 
as a machine that could be used for many other purposes. 

Before Whirlwind, all digitai computers operated as batch processors where a 
programmer creates a program (typically described using a stack of punch cards) 
and submits it to thè computer. A computer operator would set up thè computer 
to run thè program, after which it would run and (hopefully) produce a result. A 
flight simulator, though, requires direct interaction between a human user and 
thè computer. 

The first Whirlwind computer was designed in 1947 and operational by 1950. It 
was thè first interactive programmable digitai computer. Producing a machine 
that could perform thè complex computations needed for a flight simulator fast 
enough to be used interactively required much faster and more reliable mem- 
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ory that was possible with available technologies based on storing electrostatic 
charges in vacuum tubes. Jay Forrester invented a much faster memory based 
known as magnetic-core memory. Magnetic-core memory Stores a bit using 
magnetic polarity. 

The interactiveness of thè Whirlwind computer opened up many new possibili- 
ties for computing. Shortly after thè first Whirlwind computer, Ken Olson led an 
effort to build a version of thè computer using transistors. The successor to this 
machine became thè TX-2, and Ken Olsen went on to found Digital Equipment 
Corporation (DEC) which pioneered thè widespread use of moderately priced 
computers in Science and industry. DEC was very successful in thè 1970s and 
1980s, but suffered a long decline before eventually being bought by Compaq. 

Ivan Sutherland, then a graduate student at MIT, had an opportunity to use thè 
TX-2 machine. He developed a program called Sketchpad that was thè first pro- 
gram to have an interactive graphical interface. Sketchpad allowed users to draw 
and manipulate objects on thè screen using a light pen. It was designed around 
objects and operations on those objects: 3 

In thè process of making thè Sketchpad System operate, a few very gen- 
erai functions mere developed which make no reference at all to thè spe- 
cific types of entities on which they operate. These generai functions give 
thè Sketchpad System thè ability to operate on a wide range of problems. 

The motivation for making thè functions as generai as possible carne from 
thè desire to get as much result as possible from thè programming effort 
involved . . . Each ofthe generai functions implemented in thè Sketchpad 
System abstracts, in some sense, some common property ofpictures inde- 
pendent ofthe specifìc subject matter ofthe pictures themselves. 

Sketchpad was a great influence on Douglas Engelbart who developed a research 
program around a vision of using computers interactively to enhance human in- 
tellect. In what has become known as “thè mother of all demos”, Engelbart and 
his colleagues demonstrated a networked, graphical, interactive computing Sys- 
tem to thè generai public for thè first time in 1968. With Bill English, Engelhard 
also invented thè computer mouse. 

Sketchpad also influenced Alan Kay in developing object-oriented programming. 
The first language to include support for objects was thè Simula programming 
language, developed in Norway in thè 1960s by Kristen Nygaard and Ole Johan 
Dahl. Simula was designed as a language for implementing simulations. It pro- 
vided mechanisms for packaging data and procedures, and for implementing 
subclasses using inheritance. 

In 1966, Alan Kay entered graduate school at thè University of Utah, where Ivan 
Sutherland was then a professor. Here’s how he describes his first assignment: 4 

Head whirling, I found my desk. On it was a pile of tapes and listings, 
and a note: “This is thè Algol for thè 1108. It doesn’t work. Please make it 
work.” The latest graduate student gets thè latestdirty task. The documen- 
tation was incomprehensible. Supposedly, this was thè Case-Western Re- 
serve 1 1 07 Algol — but it had been doctored to make a language called Sim- 
ula; thè documentation read like Norwegian transliterated into English, 

3 Ivan Sutherland, Sketchpad: a Man-Machine Graphical Communication System, 1963 

4 AlanKay, The Early History ofSmalltalk, 1993 
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which infact it was. There were uses ofwords like activity and process that 
didn’t seem to coincide with normal English usage. Finally, another grad- 
uate student and I unrolled thè program listing 80feet down thè hall and 
crawled over ityelling discoveries to each other. The weirdest part was thè 
Storage allocator, which did not obey a stack discipline as was usuai for Al- 
gol. A few days later, that provided thè clue. What Simula was allocating 
were structures veiy much like thè instances ofSketchpad. There were de- 
scriptions that acted like masters and they could create instances, each of 
which wasan independent entity. . . . 

This was thè big hit, and I’ve not been thè same since. . . For thefirst dine 
I thought ofthe whole as thè entire computer and wondered why anyone 
would want to divide it up into weaker things called data structures and 
procedures. Why not divide it up into little computers, as time sharing was 
starting to? But not in dozens. Why not thousands ofthem, each simulat- 
ila useful structure? 

Alan Kay went on to design thè language Smalltalk, which became thè first widely 
used object-oriented language. Smalltalk was developed as part of a project at 
XEROX’s Palo Alto Research Center to develop a hand-held computer that could 
be used as a learning environment by children. 

In Smalltalk, everything is an object, and all computation is done by sending 
messages to objects. For example, in Smalltalk one computes (+ 1 2) by send- 
ing thè message + 2 to thè object 1 . Here is Smalltalk code for implementing a 
counter object: 

class name counter 
instance variable names count 
new count <— 0 
next count <— count + 1 
current ~ count 

The new method is a constructor analogous to make-counter . The count in- 
stance variable Stores thè current value of thè counter, and thè next method up- 
dates thè counter value by sending thè message + 1 to thè count object. 

Nearly all widely-used languages today provide built-in support for some form 
of object-oriented programming. For example, here is how a counter object 
could be defined in Python: 

class counter. 

def init (self: self, count = 0 
def resl(selj): self. count - 0 
def next{selfì : self. count- self. count + 1 
def currentiselfi : return self. .count 

The constructor is named __ init _ Similarly to thè object System we developed 
for Scheme, each method takes thè self object as its parameter. 

10.4 Summary 

An object is an entity that packages state with procedures that manipulate that 
state. By packaging state and procedures together, we can encapsulate state in 
ways that enable more elegant and robust programs. 



Alan Kay 


Doni worry about 
what anybody else is 
goingtodo. Thebest 
way to predict thè 
future is to invent it. 
Ready smart people 
with reasonable 
funding can do just 
about anything that 
doesn’t violate too 
many ofNewton's 
Laws! 

Alan Kay 
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10.4. Summary 


Inheritance allows an implementation of one class to reuse or override meth- 
ods in another class, known as its superclass. Programming using objects and 
inheritance enables a style of problem solving known as object-oriented pro- 
gramming in which we solve problems by modeling problem instances using 
objects. 


The Dyn&Book 

"I vish to God thcso calculations were 
exocutcd by steami" 

Charles Dabbage (age 19) 
ca. 1803 

"The Anaiytical Engine veaves 
algebralc patterns, just as thè 
Jacquard boom veaves patterns in 
silk." 

-Ada Augusta 
Countess of Lovelace 




Dynabook Images 

From Alan Kay , A Personal Computer for Children ofAHAges, 1972. 



Interpreters 


"Wlien I use a word,” Humpty Dumpty said, in a rather scornful tone, “it means just whatl choose it to 
mean - nothing more nor less.” 

“The question is,” said Alice, “whetheryou can make words mean so many different things.” 

Lewis Carroll, Through thè Looking Glass 


The tools we use have a profound (and devious!) influence on our 
thinking habits, and, therefore, on our thinking abili ties. 
Edsger Dijkstra, Hoiv do we teli truths that might hurt? 

Languages are powerful tools for thinking. Different languages encourage dif- 
ferent ways of thinking and lead to different thoughts. Hence, inventing new 
languages is a powerful way for solving problems. We can solve a problem by 
designing a language in which it is easy to express a solution and implementing 
an interpreter for that language. 

An interpreter is just a program. As input, it takes a specification of a program in 
some language. As output, it produces thè output of thè input program. Imple- 
menting an interpreter further blurs thè line between data and programs, that 
we fìrst crossed in Chapter 3 by passing procedures as parameters and return- 
ing new procedures as results. Programs are just data input for thè interpreter 
program. The interpreter determines thè meaning of thè program. 

To implement an interpreter for a given target language we need to: 

1. Implement a parser that takes as input a string representation of a pro- 
gram in thè target language and produces a structural parse of thè input 
program. The parser should break thè input string into its language com- 
ponents, and form a parse tree data structure that represents thè input text 
in a structural way. Section 11.2 describes our parser implementation. 

2. Implement an evaluator that takes as input a structural parse of an input 
program, and evaluates that program. The evaluator should implement 
thè target language’s evaluation rules. Section 1 1.3 describes our evaluator. 

Our target language is a simple subset of Scheme we cali Charme. 1 The Charme 
language is very simple, yet is powerful enough to express all computations (that 
is, it is a universal programming language). Its evaluation rules are a subset of 

Dite originai name of Scheme was “Schemer”, a successor to thè languages “Planner” and “Con- 
niver”. Because thè computer on which “Schemer” was implemented only allowed six-letter file 
names, its name was shortened to “Scheme”. In that spirit, we name our snake-charming language, 
“Charmer” and shorten it to Charme. Depending on thè programmer’s state of mind, thè language 
name can be pronounced either “charm” or “citar me”. 


interpreter 


parser 


evaluator 
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thè stateful evaluation rules for Scheme. The full grammar and evaluation rules 
for Charme are given in Section 11.3. The evaluator implements those evalua- 
tion rules. 

Section 11.4 illustrates how changing thè evaluation rules of our interp reter opens 
up new ways of programming. 


11.1 Python 

We could implement a Charme interpreter using Scheme or any other univer- 
sal programming language, but implement it using thè programming language 
Python. Python is a popular programming language initially designed by Guido 
van Rossum in 1991. 2 Python is freely available from http://www.python.org. 

We use Python instead of Scheme to implement our Charme interpreter for a few 
reasons. The hrst reason is pedagogical: it is instructive to learn new languages. 

J . As Dijkstra’s quote at thè beginning of this chapter observes, thè languages we 

■ use have a profound effect on how we think. This is true for naturai languages, 

but also true for programming languages. Different languages make different 
■ styles of programming more convenient, and it is important for every program- 

i mer to be familiar with several different styles of programming. All of thè major 

■ concepts we have covered so far apply to Python nearly identically to how they 

apply to Scheme, but seeing them in thè context of a different language should 
make it clearer what thè fundamental concepts are and what are artifacts of a 
Kwl- particular programming language. 

Another reason for using Python is that it provides some features that enhance 
expressiveness that are not available in Scheme. These include built-in support 
for objects and imperative control structures. Python is also well-supported by 
most web servers (including Apache), and is widely used to develop dynamic 
web applications. 


python 

A 

powered 


The grammar for Python is quite different from thè Scheme grammar, so Python 
programs look very different from Scheme programs. The evaluation rules, how- 
ever, are quite similar to thè evaluation rules for Scheme. This chapter does 
not describe thè entire Python language, but introduces thè grammar rules and 
evaluation rules for thè most important Python constructs as we use them to 
implement thè Charme interpreter. 

Like Scheme, Python is a universal programming language. Both languages can 
express all mechanical computations. For any computation we can express in 
Scheme, there is a Python program that defines thè same computation. Con- 
versely, every Python program has an equivalent Scheme program. 


One piece of evidence that every Scheme program has an equivalent Python 
program is thè interpreter we develop in this chapter. Since we can implement 
an interpreter for a Scheme-like language in Python, we know we can express 
every computation that can be expressed by a program in that language with an 
equivalent Python program: thè Charme interpreter with thè Charme program 
as its input. 


Tokenizing. We introduce Python using one of thè procedures in our inter- 


2 The name Python alludes to Monty Python’s Flying Circus. 
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preter implementation. We divide thè job of parsing into two procedures that 
are combined to solve thè problem of transforming an input string into a list de- 
scribing thè input program’s structure. The first part is thè tokenizer. It takes as tokenizer 
input a string representing a Charme program, and outputs a list of thè tokens 
in that string. 

A token is an indivisible syntactic unit. For example, thè Charme expression, token 
(define square (lambda (x) (* x x))), contains 15 tokens: (, define, square, (, 
lambda, (, x, ), (, *, x, x, ), ), and ). Tokens are separated by whitespace (spaces, 
tabs, and newlines). Punctuation marks such as thè left and right parentheses 
are tokens by themselves. 

The tokenize procedure below takes as input a string s in thè Charme target lan- 
guage, and produces as output a list of thè tokens in s. We describe thè Python 
language constructs it uses next. 


def tokenize(s): # ttstartsacommentuntiltheendoftheline 

Clirrent =' ' # initialìze current to thè empty string (two single quotes) 


tokens- [] # 
for c in s: # 
if c.isspaceQ : # 
if len(current) > 0: # 
tokens.append(current) # 
current-" # 
elifcin'O': # 
if len(current) > 0: # 

lokens. appendi currenl) # 
current = ' ' # 
tokens.appendic ) # 
else: # 

current = current + c # 

# end of thè for loop 
if lenicurrent ) > 0: # 
tokens.append{current) # 
return tokens # 


initialize tokens to thè empty list 
for each character, c, in thè string s 
if c is a whitespace 
ifthe current token is non-empty 
add it to thè list 
reset current token to empty string 
otherwise, if c is a parenthesis 
end thè current token 
add it to thè tokens list 
and reset current to thè empty string 
add thè parenthesis to thè token Usi 
otherwise (it is an alphanumeric) 
add thè character to thè current token 
reached thè end o/s 
ifthere is a current token 
add it to thè token list 
thè result is thè list of tokens 


11.1.1 Python Programs 

Whereas Scheme programs are composed of expressions and definitions, Python 
programs are mostly sequences of statements. Unlike expressions, a statement 
has no value. The emphasis on statements impacts thè style of programming 
used with Python. It is more imperative than that used with Scheme: instead 
of composing expressions in ways that pass thè result of one expression as an 
operand to thè next expression, Python procedures consist mostly of statements, 
each of which alters thè state in some way towards reaching thè goal state. Nev- 
ertheless, it is possible (but not recommended) to program in Scheme using an 
imperative style (emphasizing assignments), and it is possible (but not recom- 
mended) to program in Python using a functional style (emphasizing procedure 
applications and eschewing statements). 

Defining a procedure in Python is similar to defìning a procedure in Scheme, 
except thè syntax is different: 
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ProcedureDefìnition 

Parameters 

Parameters 

SomeParameters 

SomeParameters 


=> def Name ( Parameters ) : Block 

=>- e 

=> SomeParameters 
=> Name 

Name , SomeParameters 


Block 

Block 

Statements 

MoreStatements 

MoreStatements 


=>- Statement 

=> <newline> indented[Statements) 

=> Statement <newline> MoreStatements 
=> Statement <newllne> MoreStatements 

=> e 


Unlike in Scheme, whitespace (such as new lines) has meaning in Python. State- 
ments cannot be separated into multiple lines, and only one statement may ap- 
pear on a single line. Indentation within a line also matters. Instead of using 
parentheses to provide code structure, Python uses thè indentation to group 
statements into blocks. The Python interpreter reports an error if thè inden- 
tation does not match thè logicai structure of thè code. 

Since whitespace matters in Python, we include newlines (<newline>) and in- 
dentation in our grammar. We use indented{elements) to indicate that thè ele- 
ments are indented. For example, thè rule for Block is a newline, followed by one 
or more statements. The statements are all indented one level inside thè block’s 
indentation. The block ends when thè indenting returns to thè outer level. 

The evaluation rule for a procedure defìnition is similar to thè rule for evaluating 
a procedure defìnition in Scheme. 

Python Procedure Defìnition. The procedure defìnition, 

def Name ( Parameters ): Block 

defines Name as a procedure that takes as inputs thè Parameters and 
has thè body expression Block. 

The procedure defìnition, def tokenizeis): ..., defines a procedure named tokenize 
that takes a single parameter, s. 

Assignment. The body of thè procedure uses several different types of Python 
statements. Following Python’s more imperative style, fìve of thè statements in 
tokenize are assignment statements including thè fìrst two statements. For ex- 
ample, thè assignment statement, tolcens = [] assigns thè value [] (thè empty list) 
to thè name tokens. 

The grammar for thè assignment statement is: 

Statement ::=> AssignmentStatement 

AssignmentStatement ::=*► Target = Expression 
Target ::=> Name 

For now, we use only a Name as thè left side of an assignment, but since other 
constructs can appear on thè left side of an assignment statement, we introduce 
thè nonterminal Target for which additional rules can be defined to encompass 
other possible assignees. Anything that can hold a value (such as an element of 
a list) can be thè target of an assignment. 


Chapterll. Interpreters 


215 


The evaluation rule for an assignment statement is similar to Scheme’s evalua- 
tion rule for assignments: thè meaning of x = e in Python is similar to thè mean- 
ing of (set! x e) in Scheme, except that in Python thè target Name need not exist 
before thè assignment. In Scheme, it is an error to evaluate (set! x 7) where thè 
name x has not been previously defined; in Python, if x is not already defìned, 
evaluating x - 7 creates a new place named x with its value initialized to 7. 

Python Evaluation Rule: Assignment. To evaluate an assignment state- 
ment, evaluate thè expression, and assign thè value of thè expression to 
thè place identified by thè target. If no such place exists, create a new 
place with that name. 


Arithmetic and Comparison Expressions. Python supports many different 
kinds of expressions for performing arithmetic and comparisons. Since Python 
does not use parentheses to group expressions, thè grammar provides thè group- 
ing by breaking down expressions in several steps. This defìnes an order of prece- 
dere for parsing expressions. 

For example, consider thè expression 3 + 4*5. In Scheme, thè expressions (+ 
3 (* 4 51) and (* (+ 3 4) 5) are clearly different and thè parentheses group thè 
subexpressions. The Python expression, 3 + 4*5, means (+ 3 (* 4 5)) and evalu- 
ates to 23. 


Supporting precedence makes thè Python grammar rules more complex since 
they must deal with * and + differently, but it makes thè meaning of Python ex- 
pressions match our familiar mathematical interpretation, without needing to 
clutter expressions with parentheses. This is done is by defming thè grammar 
rules so an AddExpression can contain a MultExpression as one of its subexpres- 
sions, but a MultExpression cannot contain an AddExpression. This makes thè 
multiplication operator have higher precedence than thè addition operator. If an 
expression contains both + and * operators, thè * operator is grouped with its 
operands fìrst. The replacement rules that happen fìrst have lower precedence, 
since their components must be built from thè remaining pieces. 


Here are thè grammar rules for Python expressions for comparison, multiplica- 
tion, and addition expressions: 


Expression 

CompExpr 

Comparator 

CompExpr 


=> CompExpr 

=> CompExpr Comparator CompExpr 

=> < | > | == | <= | >= 

AddExpression 


AddExpression 

AddExpression 

AddExpression 


=? AddExpression + MultExpression 
=> AddExpression - MultExpression 
=> MultExpression 


MultExpression ::=> MultExpression * PrimaryExpression 

MultExpression ::=> PrimaryExpression 


PrimaryExpression ::=? 
PrimaryExpression ::=>• 
PrimaryExpression 


Literal 

Name 

( Expression ) 


precederne 
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The last rule allows expressions to be grouped explicitly using parentheses. For 
example, (3 + 4) * 5 is parsed as thè PrimaryExpression, (3 + 4), times 5, so evalu- 
ates to 35; without thè parentheses, 3 + 4 * 5 is parsed as 3 plus thè MultExpres- 
sion, 4 * 5, so evaluates to 23. 

A PrimaryExpression can be a Literal, such as a number. Numbers in Python are 
similar (but not identical) to numbers in Scheme. 

A PrimaryExpression can also be a name, similar to names in Scheme. The eval- 
uation rule for a name in Python is similar to thè stateful rule for evaluating a 
name in Scheme 3 . 


Exercise 11.1. Draw thè parse tree for each of thè following Python expressions 
and provide thè value of each expression. 

a. 1 + 2 + 3 * 4 

b. 3 > 2 + 2 

c. 3 * 6 >= 15 == 12 

d. (3*6 >= 15) == True 

Exercise 11.2. Do comparison expressions have higher or lower precedence 
than addition expressions? Explain why using thè grammar rules. 

11.1.2 DataTypes 

Python provides many built-in data types. We describe three of thè most useful 
data types here: lists, strings, and dictionaries. 

Lists. Python provides a list datatype similar to lists in Scheme, except instead 
of building lists from simpler parts (that is, using cons pairs in Scheme), thè 
Python list type is a built-in datatype. The other important difference is that 
Python lists are mutable like mlist from Section 9.3. 

Lists are denoted in Python using square brackets. For example, [] denotes an 
empty list and [1, 2] denotes a list containing two elements. The elements in a 
list can be of any type (including other lists). 

Elements can be selected from a list using thè list subscript expression: 

PrimaryExpression ::=> SubscriptExpression 
SubscriptExpression ::=^ PrimaryExpression [Expression] 

A subscript expression evaluates to thè element indexed by value of thè inner 
expression from thè list. For example, 

»a=[l,2,3] 

)§> n[0] => 1 

fl[l+l) => 3 

» a[3] => IndexError: list index out of range 


3 There are some subtle differences and complexities (see Section 4.1 of thè Python Reference 
Manual), however, which we do not go into here. 
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The expression p[ 0] in Python is analogous to ( car p) in Scheme. 

The subscript expression has Constant running time; unlike indexing Scheme 
lists, thè time required does not depend on thè length of thè list even if thè se- 
lection index is thè end of thè list. The reason for this is that Python Stores lists 
internally differently from how Scheme Stores as chains of pairs. The elements 
of a Python list are stored as a block in memory, so thè location of thè k th ele- 
ment can be calculated directly by adding k times thè size of one element to thè 
location of thè start of thè list. 

A subscript expression can also select a range of elements from thè list: 

SubscriptExpression ::=^ PrìmaryExpression [ Boundi 0l0 : Bound Hig /, ] 
Bound ::=^ Expression | e 

Subscript expressions with ranges evaluate to a list containing thè elements be- 
tween thè low bound and thè high bound. If thè low bound is missing, thè low 
bound is thè beginning of thè list. If thè high bound is missing, thè high bound 
is thè end of thè list. For example, 

» a= [1, 2, 3] 


> a[: 1] 

> a[ 1:] 


=► [ 1 ] 

=> [2, 3] 
=► [3] 


» «[4—2:3] 
» «[:] 


=*► [1,2,3] 


The expression p[l:] in Python is analogous to ( cdr p) in Scheme. 

Python lists are mutable (thè value of a list can change after it is created). We 
can use list subscripts as thè targets for an assignment expression: 

Target ::=> SubscriptExpression 

Assignments using ranges as targets can add elements to thè list as well as chang- 
ing thè values of existing elements: 

»a=[l,2,3] 

» a[0] = 7 

a =>• [7, 2, 3] 

» «[1:4] = [4, 5, 6] 

a =>- [7, 4, 5, 6] 

> a[l:] = [6] 

a =>- [7, 6] 

In thè tokenize procedure, we use tokens = [] to initialize tokens to an empty list, 
and use tokens.append(current ) to append an element to thè tokens list. The 
Python append procedure is similar to thè mlist-append! procedure (except it 
works on thè empty list, where there is no way in Scheme to modify thè nuli 
input list). 

Strings. The other datatype used in tokenize is thè string datatype, named str 
in Python. As in Scheme, a String is a sequence of characters. Unlike Scheme 
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strings which are mutable, thè Python str datatype is immutable. Once a string 
is createci its value cannot change. This means all thè string methods that seem 
to change thè string values actually return new strings (for example, capitalizeQ 
returns a copy of thè string with its fìrst letter capitalized) . 

Strings canbe enclosed in single quotes (e.g., 'hello'), doublé quotes (e.g., "hello"), 

and triple-double quotes (e.g., " " "hello' ; a string inside triple quotes can span 

multiple lines). In our example program, we use thè assignment expression, 
current = ' ' (two single quotes), to initialize thè value of current to thè empty 
string. The input, s, is a string object. 

The addition operator can be used to concatenate two strings. In tokenize, we 
use current - current + cto update thè value of current to include a new charac- 
ter. Since strings are immutable there is no string method analogous to thè list 
append method. Instead, appending a character to a string involves creating a 
new string object. 

Dictionaries. A dictionary is a lookup-table where values are associated with 
keys. The keys can be any immutable type (strings and numbers are commonly 
used as keys); thè values can be of any type. We did not use thè dictionary type 
in tokenize, but it is very useful for implementing frames in thè evaluator. 

A dictionary is denoted using curly brackets. The empty dictionary is {}. We 
add a key-value pair to thè dictionary using an assignment where thè left side 
is a subscript expression that specifìes thè key and thè right side is thè value 
assigned to that key. For example, 

birthyear- {} 
birthyear}' Euclid'] = '300BC' 
birthyear}' Ada'] = 1815 
birthyear}' Alan Turing'] = 1912 
birthyear}' Alan Kay'] = 1940 

defines birthyear as a dictionary containing four entries. The keys are all strings; 
thè values are numbers, except for Euclid’s entry which is a string. 

We can obtain thè value associated with a key in thè dictionary using a sub- 
script expression. For example, birthyear}' Alan Turing'] evaluates to 1912. We can 
replace thè value associated with a key using thè same syntax as adding a key- 
value pair to thè dictionary. The statement, 

birthyear}'Euc\\d'] = —300 

replaces thè value of birthyear}' Euclid'] with thè number —300. 

The dictionary type also provides a method has_key that takes one input and 
produces a Boolean indicating if thè dictionary object contains thè input value 
as a key. For thè birthyear dictionary, 

I) i ri hyea r. has key( 'John Backus') => False 
birthyear. has.keyi' Ada') => True 


The dictionary type lookup and update operations have approximately Constant 
running time: thè time it takes to lookup thè value associated with a key does not 
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scale as thè size of thè dictionary increases. This is done by computing a number 
based on thè key that determines where thè associated value would be stored (if 
that key is in thè dictionary) . The number is used to index into a structure similar 
to a Python list (so it has Constant time to retrieve any element). Mapping keys 
to appropriate numbers to avoid many keys mapping to thè same location in 
thè list is a diffìcult problem, but one thè Python dictionary object does well for 
most sets of keys. 

11.1.3 Applications and Invocations 

The grammar rules for expressions that apply procedures are: 


PrimaryExpression 
CallExpression ::=>- 
ArgumentList ::=> 
ArgumentList ::=>- 
SomeArguments ::=>- 
SomeArguments ::=>- 


CallExpression 

PrimaryExpression ( ArgumentList ) 

SomeArguments 

e 

Expression 

Expression , SomeArguments 


In Python, nearly every data value (including lists and strings) is an object. This 
means thè way we manipulate data is to invoke methods on objects. To invoke a 
method we use thè same rules, but thè PrimaryExpression of thè CallExpression 
specifies an object and method: 

PrimaryExpression ::=>- AttributeReference 
AttributeReference ::=> PrimaryExpression . Name 

The name AttributeReference is used since thè same syntax is used for accessing 
thè internai state of objects as well. 

The tokenize procedure includes five method applications, four of which are 
tokens.append{current). The object reference is tokens, thè list of tokens in thè 
input. The list append method takes one parameter and adds that value to thè 
end of thè list. 

The other method invocation is c. isspace () where c is a string consisting of one 
character in thè input. The isspace method for thè string datatype returns true 
if thè input string is non-empty and all characters in thè string are whitespace 
(either spaces, tabs, or newlines). 

The tokenize procedure also uses thè built-in function len which takes as in- 
put an object of a collection datatype such as a list or a string, and outputs thè 
number of elements in thè collection. It is a procedure, not a method; thè input 
object is passed in as a parameter. In tokenize, we use len(current) to fìnd thè 
number of characters in thè current token. 

11.1.4 Control Statements 

Python provides control statements for making decisions, looping, and for re- 
turning from a procedure. 

If statement. Python’s if statement is similar to thè conditional expression in 
Scheme: 
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Statement ::=> IfStatement 

li Statement ::=> if Expression Predicate : Block Elifs OptElse 
Elifs ::=> e 

Elifs ::=> elif Expression p r( , dlcate : Block Elifs 

OptElse ::=> e 

OptElse ::=r else : Block 


Unlike in Scheme, there is no need to have an alternate clause since thè Python 
if statement does not need to produce a value. The evaluation rule is similar to 
Scheme’s conditional expression: 

Python Evaluation Rule: If. First, evaluate thè Expression Predicate . If it 
evaluates to a true value, thè consequent Block is evaluated, and none 
of thè rest of thè IfStatement is evaluated. Otherwise, each of thè elif 
predicates is evaluated in order. If one evaluates to a true value, its Block 
is evaluated and none of thè rest of thè IfStatement is evaluated. If none 
of thè elif predicates evaluates to a true value, thè else Block is evaluated 
if there is one. 

The main if statement in tokenize is: 

if c.isspace (): ... 

elif cin ... 

else: current- current+ c 

The fìrst if predicate tests if thè current character is a space. If so, thè end of thè 
current token has been reached. The consequent Block is itself an IfStatement : 

if len(current) > 0: 
tokens. appendi curren t) 
current - ' ' 

If thè current token has at least one character, it is appended to thè list of tokens 
in thè input string and thè current token is reset to thè empty string. This IfS- 
tatement has no elif or else clauses, so if thè predicate is false, there is nothing 
to do. 

For statement. A for statement provides a way of iterating through a set of 
values, carrying out a body block for each value. 

Statement ::=>- ForStatement 
ForStatement ::=>- for Target in Expression : Block 


Its evaluation rule is: 

Python Evaluation Rule: For. First evaluate thè Expression which must 
produce a value that is a collection. Then, for each value in thè collec- 
tion assign thè Target to that value and evaluate thè Block. 

Other than thè first two initializations, and thè final two statements, thè bulk 
of thè tokenize procedure is contained in a for statement. The for statement in 
tokenize header is for cin s: .... The string 5 is thè input string, a collection of 
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characters. So, thè loop will repeat once for each character in s, and thè value of 
cis each character in thè input string (represented as a singleton stringi, in turn. 

Return statement. In Scheme, thè body of a procedure is an expression and thè 
value of that expression is thè result of evaluating an application of thè proce- 
dure. In Python, thè body of a procedure is a block of one or more statements. 
Statements have no value, so there is no obvious way to decide what thè result 
of a procedure application should be. Python’s solution is to use a return state- 
ment. 

The grammar for thè return statement is: 

Statement ::=>- ReturnStatement 
ReturnStatement ::=> return Expression 

A return statement fmishes execution of a procedure, returning thè value of thè 
Expression to thè caller as thè result. The last statement of thè tokenize proce- 
dure is: return tokens. It returns thè value of thè tokens list to thè caller. 


11.2 Parser 

The parser takes as input a Charme program string, and produces as output a 
nested list that encodes thè structure of thè input program. The first step is to 
break thè input string into tokens; this is done by thè tokenize procedure defìned 
in thè previous section. 

The next step is to take thè list of tokens and produce a data structure that en- 
codes thè structure of thè input program. Since thè Charme language is built 
from simple parenthesized expressions, we can represent thè parsed program 
as a list. But, unlike thè list returned by tokenize which is a fiat list containing 
thè tokens in order, thè list returned by parse is a structured list that may have 
lists (and lists of lists, etc.) as elements. 


Charme’s syntax is very simple, so thè parser can be implemented by just break- 
ing an expression into its components using thè parentheses and whitespace. 
The parser needs to balance thè open and dose parentheses that enclose ex- 
pressions. For example, if thè input string is 

(define square (lambda (x) (* x x))) 


thè output of tokenizer is thè list: 


['(', 'define', 'square', ’(', 'lambda', '(’, 'x', ')', '(', Y, 'x', ')’, ')’, ')’] 


The parser structures thè tokens according to thè program structure, producing 
a parse tree that encodes thè structure of thè input program. The parenthesis 
provide thè program structure, so are removed from thè parse tree. For thè ex- 
ample, thè resulting parse tree is: 

['define', 

'square', 

[ 'lambda', 
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Here is thè definition of parse : 
def parse(s) : 

def parseJokens{tokens, inner): 
res= [] 

while len[tokens ) > 0: 
current- tokens.popiO) 
if current == 

res.append ( parseJokens[tokens , True)) 
elif current-- ')’: 
if inner. return res 
else: 

errorCUnmatched dose pareri: ' + s) 
return None 
else: 

res. appendi current) 
if inner. 

error ('Unmatched open paren: ' + s) 
return None 
else: 

return res 

return parseJokens) lokenize(s) , False) 

The input to parse is a string in thè target language. The output is a list of thè 
parenthesized expressions in thè input. Here are some examples: 

» parse ('150') => ['150'] 

> parseC{+ 1 2)’) [['+', '1', '2']] 

» parseC(+ 1 (* 2 3))') =► [['+', '1', ['*', '2', '3']]] 

» parse {'( define square (lambda (x) (* x x)))’) 

=> [['define', 'square', ['lambda', ['x'], ['*', ’x’, 'x']]]] 

> parsei' {+ 1 2) (+ 3 4)') =► [['+', T, '2'], ['+', '3', '4']] 

The parentheses are no longer included as tokens in thè result, but their pres- 
ence in thè input string determines thè structure of thè result. 

descent The parse procedure implements a recursive descent parser. The main parse 
procedure defìnes thè parseJokens helper procedure and returns thè result of 
calling it with inputs that are thè result of tokenizing thè input string and thè 
Boolean literal False: return parse Jokens{tokenize{s), False). 

The parseJokens procedure takes two inputs: tokens, a list of tokens (that results 
from thè tokenize procedure); and inner, a Boolean that indicates whether thè 
parser is inside a parenthesized expression. The value of inner is False for thè 
initial cali since thè parser starts outside a parenthesized expression. All of thè 
recursive calls result from encountering a ’(', so thè value passed as inner is True 
for all thè recursive calls. 

The body of thè parseJokens procedure initializes res to an empty list that is 
used to store thè result. Then, thè while statement iterates as long as thè token 
list contains at least one element. 
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The first statement of thè while statement block assigns tokens.popl 0) to current. 
The pop method of thè list takes a parameter that selects an element from thè 
list. The selected element is retumed as thè result. The pop method also mutates 
thè list object by removing thè selected element. So, lokens.popdì) returns thè 
first element of thè tokens list and removes that element from thè list. This is 
essential to thè parser making progress: every time thè tokens.popl 0) expression 
is evaluated thè number of elements in thè token list is reduced by one. 

If thè current token is an open parenthesis, parse_tokens is called recursively to 
parse thè inner expression (that is, all thè tokens until thè matching dose paren- 
thesis). The result is a list of tokens, which is appended to thè result. If thè 
current token is a dose parenthesis, thè behavior depends on whether or not 
thè parser is parsing an inner expression. If it is inside an expression (that is, 
an open parenthesis has been encountered with no matching dose parenthesis 
yet), thè dose parenthesis closes thè inner expression, and thè result is returned. 
If it is not in an inner expression, thè dose parenthesis has no matching open 
parenthesis so a parse errar is reported. 

The else clause deals with all other tokens by appending them to thè list. 

The final if statement checks that thè parser is not in an inner context when thè 
input is finished. This would mean there was an open parenthesis without a 
corresponding dose, so an error is reported. Otherwise, thè list representing thè 
parse tree is returned. 

11.3 Evaluator 

The evaluator takes a list representing thè parse tree of a Charme expression or 
definition and an environment, and outputs thè result of evaluating thè expres- 
sion in thè input environment. The evaluator implements thè evaluation rules 
for thè target language. 

The core of thè evaluator is thè procedure meval : 
def mevaUexpr, env): 

if is primitive[expr) : return e vai p rimi li ve{ ex pr) 

elif isJfiexpr ): return evaLiJlexpr, env) 

elif is_ de fi n i t io n{ ex p r) : e va Ldefi n i l ioti ( exp r } env) 

elif isjiame(expr): return evaLname(expr, env) 

elif isJambda{expr): return e vai lambda) expr, env) 

elif is application(expr) : return e va l ap p l i ca t io n ( exp r, env) 

else: error (’Unknown expression type: ' + str{expr)) 

The if statement matches thè input expression with one of thè expression types 
(or thè definition) in thè Charme language, and returns thè result of applying 
thè corresponding evaluation procedure (if thè input is a definition, no value is 
returned since definitions do not produce an output value). We next consider 
each evaluation rule in turn. 

11.3.1 Primitives 

Charme supports two kinds of primitives: naturai numbers and primitive pro- 
cedures. If thè expression is a number, it is a string of digits. The is.number 
procedure evaluates to True if and only if its input is a number: 
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def is. p rii n ilive(expr) : 

return is number(expr) or is. primitive procedure(expr) 
def is.number(expr ): 

return isinstance{expr, str) and expr.isdigitQ 

Here, we use thè built-in function isinstance to check if expr is of type str. The 
and expression in Python evaluates similarly to thè Scheme and special form: 
thè left operand is evaluated first; if it evaluates to a false value, thè value of 
thè and expression is that false value. If it evaluates to a true value, thè right 
operand is evaluated, and thè value of thè and expression is thè value of its right 
operand. This evaluation rule means it is safe to use expr. isdigit{ ) in thè right 
operand, since it is only evaluated if thè left operand evaluated to a true value, 
which means expr is a string. 

Primitive procedures are defined using Python procedures. We defìne thè pro- 
cedure is .primitive .procedure using callable, a procedure that returns true only 
for callable objects such as procedures and methods: 

def is_primitive_procedure{expr)\ 
return callable{expr) 

The evaluation rule for a primitive is identical to thè Scheme rule: 

Charme Evaluation Rule 1: Primitives. A primitive expression evalu- 
ates to its pre-defined value. 

We need to implement thè pre-defined values in our Charme interpreter. 

To evaluate a number primitive, we need to convert thè string representation 
to a number of type int. The ini{s) constructor takes a string as its input and 
outputs thè corresponding integer: 

def eval_primitive{expr ): 
if is nurnber(expr): return int[expr ) 
else: return expr 

The else clause means that all other primitives (in Charme, this is only primi- 
tive procedures and Boolean constants) self-evaluate: thè value of evaluating a 
primitive is itself. 

For thè primitive procedures, we need to defìne Python procedures that imple- 
ment thè primitive procedure. For example, here is thè primitive.plus procedure 
that is associated with thè + primitive procedure: 

def primitive.plus ( operands ): 
if ( leni operands) -- 0): return 0 

else: return operands[ 0] + primitive.plus (opemnds[ 1 :]) 

The input is a list of operands. Since a procedure is applied only after all subex- 
pressions are evaluated, there is no need to evaluate thè operands: they are ai- 
ready thè evaluated values. For numbers, thè values are Python integers, so we 
can use thè Python + operator to add them. To provide thè same behavior as thè 
Scheme primitive + procedure, we defìne our Charme primitive + procedure to 
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evaluate to 0 when there are no operands, and otherwise to recursively add all 
of thè operand values. 

The other primitive procedures are defìned similarly: 

def primitive Jimes [operands)-. 
if ( len[operands ) == 0): return 1 

else: return operands) 0] * primitive Limes [operands) 1:]) 

def primitivejninus [operands)-. 
if [len[operands) == 1): return —1 * operands) 0] 
elif len[operands) == 2 : return operands)0] — operands [1] 

else: 

evaLerror['~ expects 1 or 2 operands, given %s: %s' 

% [len[operands) , str[operands))) 

def primitive_equals [operands): 
check_operands [operands, 2, '=') 
return operands) 0] == operands) 1] 

def primitive Jessthan [operands): 
check.operands [operands, 2, '<’) 
return operands) 0] < operands) 1] 

The check-operands procedure reports an error if a primitive procedure is ap- 
plied to thè wrong number of operands: 

def check_operands[operands, num, prim): 
if [len[operands) != num): 

evaLerror[' Primitive %s expected %s operands, given %s: %s' 

% [prim, num, len[operands) , str[operands))) 

11.3.2 If Expressions 

Charme provides an if expression special form with a syntax and evaluation rule 
identical to thè Scheme if expression. The grammar rule for an if expression is: 

IfExpression ::=^ (if Expression Piedìcate 
Expression Conse q Uent 

Expression^ Iternate) 


The expression object representing an if expression should be a list containing 
three elements, with thè first element matching thè keyword if. 

All special forms have this property: they are represented by lists where thè first 
element is a keyword that identifies thè special form. 

The is_special_form procedure takes an expression and a keyword and outputs 
a Boolean. The result is True if thè expression is a special form matching thè 
keyword: 

def isspeciaLform[expr, keyword) : 

return isinstance[expr, list) and len[expr) > 0 and expr) 0] == keyword 
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We can use this to recognize different special forms by passing in different key- 
words. We recognize an if expression by thè if token at thè beginning of thè ex- 
pression: 

def is ifiexpr) : 

return is special form(expr, 'if') 

The evaluation rule for an if expression is: 4 

Charme Evaluation Rule 5: If. To evaluate an if expression in thè cur- 
rent environment, (a) evaluate thè predicate expression in thè current 
environment; then, (b) if thè value of thè predicate expression is a false 
value then thè value of thè if expression is thè value of thè alternate ex- 
pression in thè current environment; otherwise, thè value of thè if ex- 
pression is thè value of thè consequent expression in thè current envi- 
ronment. 

This procedure implements thè if evaluation rule: 
def evaLiJ{expr,env): 

if meval{expr[ 1], env) != False: return meval{expr[2],env) 
else: return meval[expr[3],env) 

11.3.3 Defìnitions and Names 

To evaluate defìnitions and names we need to represent environments. A def- 
inition adds a name to a frame, and a name expression evaluates to thè value 
associated with a name. 

We use a Python class to represent an environment. As in Chapter 10, a class 
packages state and procedures that manipulate that state. In Scheme, we needed 
to use a message-accepting procedure to do this. Python provides thè class 
construct to support it directly. We defìne thè Environment class for represent- 
ing an environment. It has internai state for representing thè parent (itself an 
Environment or None, Python’s equivalent to nuli for thè global environment’s 
parent), and for thè frame. 

The dictionary datatype provides a convenient way to implement a frame. The 
Jnit__ procedure constructs a new object. It initializes thè frame of thè new 
environment to thè empty dictionary using self. .frame = {}. 

The add-variable method either defìnes a new variable or updates thè value as- 
sociated with a variable. With thè dictionary datatype, we can do this with a 
simple assignment statement. 

The lookup-variable method fìrst checks if thè frame associated with this envi- 
ronment has a key associated with thè input name. If it does, thè value associ- 
ated with that key is thè value of thè variable and that value is returned. Other- 
wise, if thè environment has a parent, thè value associated with thè name is thè 
value of looking up thè variable in thè parent environment. This directly follows 
from thè stateful Scheme evaluation rule for name expressions. The else clause 

4 We number thè Charme evaluation rules using thè numbers we used for thè analogous Scheme 
evaluation rules, but present them in a different order. 
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addresses thè situation where thè name is not found and there is no parent en- 
vironment (since we have already reached thè global environment) by reporting 
an evaluation error indicating an undefined name. 

class Environment: 
def init (self parent ): 
self, parent = parent 
self. -frante - {} 

def add_ va rial) le{ self name, value ): 
self._frame[name] = value 

def looìcup-variable[self name): 
if self. J'rame. has_key( name) : return self._frame[name ] 
elif (self -parent): return self-parent.lookup-variable{name ) 
else: ei/aLenw('Undefined name: %s' % (name)) 

Using thè Environment class, thè evaluation rules for defìnitions and name ex- 
pressions are straightforward. 

def is_definition(expr): return is_special_form(expr, 'define') 
def evaLdefìnition(expr, env): 
name = expr[ 1] 
value = meval(expr[ 2], env) 
env.add_variable(name, value) 

def is_name(expr): return isinstance(expr, str) 
def evaLname(expr, env): 
return env. lookup-variable(expr) 

11.3.4 Procedures 

The result of evaluating a lambda expression is a procedure. Hence, to define thè 
evaluation rule for lambda expressions we need to define a class for representing 
user-defmed procedures. It needs to record thè parameters, procedure body, 
and defming environment: 

class Procedure-. 

def _ Jnit— (self params, body, env): 
self-params = params 
self. -body- body 
self-env - env 

def getParams(selfl : return self -params 
def getBody(selfi : return self. -body 
def getEnvironment(selfl: return self-env 

The evaluation rule for lambda expressions creates a Procedure object: 
def isJambda(expr): return isspecial_form(expr, 'lambda') 

def eval_lambda(expr,env): 
return Procedure(expr[ 1], expr\ 2], env) 
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11.3.5 Application 

Evaluation and application are defined recursively. To perform an application, 
we need to evaluate all thè subexpressions of thè application expression, and 
then apply thè result of evaluating thè first subexpression to thè values of thè 
other subexpressions. 

def iswipplicalion(expr ) : # requires: all special forms checked first 
return isinstance(expr, lisi ) 

def eval_application[expr, env): 
subexprs = expr 

subexprvals= map (lambda sexpr. mevaUsexpr, env), subexprs) 
return mapply{subexprvals[0], subexprvals [\ :]) 

The evaLapplication procedure uses thè built-in map procedure, which is sim- 
ilar to list-map from Chapter 5. The first parameter to map is a procedure con- 
structed using a lambda expression (similar in meaning, but not in syntax, to 
Scheme’s lambda expression); thè second parameter is thè list of subexpres- 
sions. 

The mapply procedure implements thè application rules. If thè procedure is a 
primitive, it “just does it”: it applies thè primitive procedure to its operands. 

To apply a constructed procedure (represented by a Procedure), follow thè state- 
ful application rule for applying constructed procedures: 

Charme Application Rule 2: Constructed Procedures. To apply a con- 
structed procedure: 

1. Construct a new environment, whose parent is thè environment 
of thè applied procedure. 

2. For each procedure parameter, create a place in thè frame of thè 
new environment with thè name of thè parameter. Evaluate each 
operand expression in thè environment or thè application and ini- 
tialize thè value in each place to thè value of thè corresponding 
operand expression. 

3. Evaluate thè body of thè procedure in thè newly created environ- 
ment. The resulting value is thè value of thè application. 

The mapply procedure implements thè application rules for primitive and con- 
structed procedures: 

def mapply) proc, operands): 
if ( is_primitive_procedure[proc )): return proc(operands) 
elif isinstanceiproc, Procedure): 
params = proc.getParams[) 
newenv = Environment[proc.getEnvironment[)) 
if leni params) != Ieri) operands) : 
evaLerror ('Parameter length mismatch: %s given operands %s' 

% ( striproc ), str[operands))) 
for (in range[ 0, leni params) ) : 

newenv. add variali le( params) i \ , operands) i\ ) 
return meval(proc. ge t B o dy ( ) , newenv) 
else: erraLerrorCApplication of non-procedure: %s' % [proc)) 
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11.3.6 Finishing thè Interpreter 

To finish thè interpreter, we defìne thè evalLoop procedure that sets up thè global 
environment and provides an interactive interface to thè interpreter. The eval- 
uation loop reads a string from thè user using thè Python built-in procedure 
raw -input. It uses parse to convert that string into a structured list representa- 
tion. Then, it uses a for loop to iterate through thè expressions. It evaluates each 
expression using meval and thè result is printed. 

To initialize thè global environment, we create an environment with no parent 
and place variables in it corresponding to thè primitives in Charme. 

def evalLoopO : 
genv = Environment{Uone) 
genv.add variable('\rue', True) 
genv.add variable{'ia\se’ , False) 
gen v. add- variable{'+', primitive.plus) 
gen v. add_ va riablel ' - ' , primitive _minus) 
gen v. add_ variableC *' , primitive _ times ) 
gen v. add_ variable('= , prim iti ve_equals) 
gen v. add va riablel '< ', primitive Jessthan) 
while True: 

inv- rawJnputl' Charme> ') 
if inv-= 'quit': break 
for expr in parse{inv)\ 
print str{meval{expr, genv)) 

Here are some sample interactions with our Charme interpreter: 

» evalLoopO 
Charme> (+ 2 2) 

4 

Charme> {define fibo 

(lambda {ri) 

(if (= n 1) 1 
(if (= n 2) 1 

(+ {fibo (- n 1)) {fibo {- n 2))))))) 

None 

Charme> {fibo 10) 

55 

11.4 Lazy Evaluation 

Once we have an interpreter, we can change thè meaning of our language by 
changing thè evaluation rules. This enables a new problem-solving strategy: if 
thè solution to a problem cannot be expressed easily in an existing language, 
define and implement an interpreter for a new language in which thè problem 
can be solved more easily. 

This section explores a variation on Charme we cali LazyCharme. LazyCharme 
changes thè application evaluation rule so that operand expressions are not 
evaluated until their values are needed. This is known as lazy evaluation. Lazy lazy evaluation 
evaluation enables many procedures which would otherwise be awkward to ex- 
press to be defined concisely. Since both Charme and LazyCharme are universal 
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programming languages they can express thè same set of computations: all of 
thè procedures we define that take advantage of lazy evaluation could be de- 
fined with eager evaluation (for example, by first defining a lazy interpreter as 
we do here). 


eager evaluation 


Mach ofmy work 
has comefrom 
being lazy. 
John Backus 


thunk 


11.4.1 Lazy Interpreter 

The Charme interpreter as well as thè standard Scheme language evaluate appli- 
cation expressions eagerly: all operand subexpressions are evaluated whether or 
not their values are needed. This is known as eager evaluation. Eager evaluation 
means that any expression that does not always evaluate all of its subexpres- 
sions must be a special form. For example, there is no way to define a procedure 
that behaves like thè if special form. 

With lazy evaluation, an expression is evaluated only when its value is needed. 
Lazy evaluation changes thè evaluation rule for applications of constructed pro- 
cedures. Instead of evaluating all operand expressions, lazy evaluation delays 
evaluation of an operand expression until thè value of thè parameter is needed. 
To keep track of what is needed to perform thè evaluation when and if it is 
needed, a special object known as a thunk is created and stored in thè place 
associated with thè parameter name. By delaying evaluation of operand expres- 
sions until their value is needed, we can enable programs to define procedures 
that conditionally evaluate their operands like thè if special form. 


We will encoumge 
you to develop thè 
tliree great virtues of 
a programmer: 
Laziness, 
Impatience, and 
Hubris. 
Larry Wall, 
Programming Perl 


The lazy rule for applying constructed procedures is: 

Lazy Application Rule 2: Constructed Procedures. To apply a con- 
structed procedure: 

1. Construct a new environment, whose parent is thè environment 
of thè applied procedure. 

2. For each procedure parameter, create a place in thè frame of thè 
new environment with thè name of thè parameter. Put a thunlc in 
that place, which is an object that can be used later to evaluate 
thè value of thè corresponding operand expression if and when 
its value is needed. 


3. Evaluate thè body of thè procedure in thè newly created environ- 
ment. The resulting value is thè value of thè application. 


The rule is identical to thè Stateful Application Rule except for thè bolded part 
of step 2. To implement lazy evaluation we modify thè interpreter to implement 
thè lazy application rule. We start by defining a Python class for representing 
thunks and then modify thè interpreter to support lazy evaluation. 

Making Thunks. A thunk keeps track of an expression whose evaluation is de- 
layed until it is needed. Once thè evaluation is performed, thè resulting value 
is saved so thè expression does not need to be re-evaluated thè next time thè 
value is needed. Thus, a thunk is in one of two possible States: unevaluated and 
evaluated. 

The Thunk class implements thunks: 
class Thunk 

def _ Jnit—{selfi expr, env): 
self.. expr = expr 
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self, .env- env 
self, .evaluated- False 
def valueiselfi : 
if not self. evaluated : 
self._value = force_evaK.self.expr, self .env) 
self, .evaluated = True 
return self. .valile 

A Thunk ob)ect keeps track of thè expression in thè _exprinstance variable. Since 
thè value of thè expression may be needed when thè evaluator is evaluating an 
expression in some other environment, it also keeps track of thè environment in 
which thè thunk expression should be evaluated in thè .env instance variable. 

The _ evaluated instance variable is a Boolean that records whether or not thè 
thunk expression has been evaluated. Initially this value is False. After thè ex- 
pression is evaluated, _ evaluated is True and thè _ value instance variable keeps 
track of thè resulting value. 

The value method uses force. e vai (dehned later) to obtain thè evaluated value of 
thè thunk expression and Stores that result in .value. 

The is.thunk procedure returns True only when its parameter is a thunk: 

def is.thunkiexpr): return isinstanceiexpr, Thunk) 

Changing thè evaluator. To implement lazy evaluation, we change thè evalua- 
tor so there are two different evaluation procedures: me vai is thè standard evalu- 
ation procedure (which leaves thunks in their unevaluated state), and force.eval 
is thè evaluation procedure that forces thunks to be evaluated to values. The 
interpreter uses meval when thè actual expression value may not be needed, 
and force.eval to force evaluation of thunks when thè value of an expression is 
needed. 

In thè meval procedure, a thunk evaluates to itself. We add a new elif clause for 
thunk objects to thè meval procedure: 

elif is.thunkiexpr): return expr 

The force.eval procedure first uses meval to evaluate thè expression normally. If 
thè result is a thunk, it uses thè Thunk. value method to force evaluation of thè 
thunk expression. That method uses force.eval to find thè value of thè thunk 
expression, so any thunks inside thè expression will be recursively evaluated. 

def force.evaUexpr, env): 
vai = mevaUexpr, env) 
if is_thunk[val ): return val.value () 

else: return vai 

Next, we change thè application rule to perform delayed evaluation and change 
a few other places in thè interpreter to use force.eval instead of meval to obtain 
thè actual values when they are needed. 

We change evaLapplication to delay evaluation of thè operands by creating Thunk 
objects representing each operand: 
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def evaLapplicationiexpr, env): 
ops - map (lambda sexpr. Thunkisexpr, env), expr[ 1:]) 
return mapplyi force eval(expr{ 0 1 , env), ops) 


Modem methods of 
production have 
given us thè 
possibility ofease 
and securityfor all; 
we have chosen, 
instead, to have 
overworkfor some 
and starnatici! for 
others. Hitherto we 
have continued to 
be as energetic as we 
were before there 
were machines; in 
this we bave been 
foolish, but there is 
no reason to go on 
being foolish 
forerei: 
Bertrand Russell, In 
Fraise ofldleness, 1932 


Only thè fìrst subexpression must be evaluated to obtain thè procedure to apply. 
Hence, evaLapplication uses force-eval lo obtain thè value ofthe first subexpres- 
sion, but makes Thunk objects for thè operand expressions. 

To apply a primitive, we need thè actual values of its operands, so must force 
evaluation of any thunks in thè operands. Hence, thè definition for mapply 
forces evaluation of thè operands to a primitive procedure: 

def mapplyi proc, operands): 
def dethunk(expr): 
if isJhunk(expr): return expr.value 0 

else: return expr 

if (is primitive procedure(proc)): 
ops = map ( dethunk , operands) 
return proc(ops) 
elif isinslanceiproc, Procedure): 

... # same as in Charme interpreter 

To evaluate an if expression, it is necessary to know thè actual value of thè pred- 
icate expressions. We change thè evaLif procedure to use force e vai when eval- 
uating thè predicate expression: 

def evaLif expr, env): 

ii force_eval{expr[\\, env) != False: return tri e va l(expr\2\,env) 
else: return meval(expr[3],env) 


This forces thè predicate to evaluate to a value so its actual value can be used 
to determine how thè rest of thè if expression evaluates; thè evaluations of thè 
consequent and alternate expressions are left as mevals since it is not necessary 
to force them to be evaluated yet. 

The final change to thè interpreter is to force evaluation when thè result is dis- 
played to thè user in thè evalLoop procedure by replacing thè cali to meval with 
force e vai 


11.4.2 Lazy Programming 

Lazy evaluation enables programming constructs that are not possible with ea- 
ger evaluation. For example, with lazy evaluation we can define a procedure 
that behaves like thè if expression special form. We first define true and false as 
procedures that take two parameters and output thè first or second parameter: 

(define true (lambda ( a b) a)) 

(define false (lambda ( a b) b)) 

Then, this definition defines a procedure with behavior similar to thè if special 
form: 


(define ifp (lambda (p c a) (p c a))) 
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With eager evaluation, this would not work sirice all operands would be evalu- 
ated; with lazy evaluation, only thè operand that corresponds to thè appropriate 
consequent or alternate expression is evaluated. 

Lazy evaluation also enables programs to deal with seemingly infinite data struc- 
tures. This is possible since only those values of thè apparently infinite data 
structure that are used need to be created. 

Suppose we define procedures similar to thè Scheme procedures for manipulat- 
ing pairs: 

(define cons (lambda (a b) (lambda (p) (if p a b) ) ) ) 

(define car (lambda (p) (p true ))) 

(define cdr (lambda (p) (p false))) 

(define nuli false) 

(define nuli? (lambda (x) (= x false))) 

These behave similarly to thè corresponding Scheme procedures, except in Lazy- 
Charme their operands are evaluated lazily. This means, we can define an infi- 
nite list: 

(define ints-from (lambda (n) ( cons n ( ints-from (+ n 1))))) 

With eager evaluation, ( ints-from 1) would never finish evaluating; it has no 
base case for stopping thè recursive applications. In LazyCharme, however, thè 
operands to thè cons application in thè body of ints-fi'om are not evaluated un- 
til they are needed. Hence, ( ints-from 1) terminates and produces a seemingly 
infinite list, but only thè evaluations that are needed are performed: 

LazyCharme> ( car ( ints-from 1)) 

1 

LazyCharme> ( car ( cdr ( cdr ( cdr ( ints-from 1))))) 

4 


Some evaluations fail to terminate even with lazy evaluation. For example, as- 
sume thè standard definition of list-length : 

(define list-length 

(lambda {Ist) (if ( nuli ? Ist) 0 (+ 1 ( list-length ( cdr Ist)))))) 

An evaluation of ( length ( ints-from 1)) never terminates. Every time an appli- 
cation of list-length is evaluated, it applies cdr to thè input list, which causes 
ints-from to evaluate another cons, increasing thè length of thè list by one. The 
actual length of thè list is infinite, so thè application of list-length does not ter- 
minate. 

Lists with delayed evaluation can be used in useful programs. Reconsider thè 
Fibonacci sequence from Chapter 7. Using lazy evaluation, we can define a list 
that is thè infmitely long Fibonacci sequence: 5 

(define flbo-gen (lambda (a b) ( cons a ( flbo-gen b (+ a b))))) 

(define fibos ( flbo-gen 0 1)) 

5 This example is based on Abelson and Sussman, Structure and Interpretation of Computer 
Programs, Section 3.5.2, which also presents several other examples of interesting programs con- 
structed using delayed evaluation. 
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The n th Fibonacci number is thè n th element of flbos: 

(define fibo 
(lambda (ri) 

(list-get- element flbos n) ) ) 

where list-get-element is defined as it was defined in Chapter 5. 

Another strategy for defining thè Fibonacci sequence is to first define a proce- 
dure that merges two (possibly infinite) lists, and then define thè Fibonacci se- 
quence recursively. The merge-lists procedure combines elements in two lists 
using an input procedure. 

(define merge-lists 
(lambda (Isti lst2 proc ) 

(if (nuli? Isti ) nuli 
(if (nuli? Ist2) nuli 

(cons (proc (car Isti ) (car lst2)) 

(merge-lists (cdr Isti ) (cdr lst2) proc)))))) 

We can define thè Fibonacci sequence as thè combination of two sequences, 
starting with thè 0 and 1 base cases, combined using addition where thè second 
sequence is offset by one position: 

(define flbos (cons 0 (cons 1 (merge-lists flbos (cdr flbos) +)))) 

The sequence is defined to start with 0 and 1 as thè first two elements. The fol- 
lowing elements are thè result of merging/zfios and (cdr flbos) using thè + pro- 
cedure. This definition relies heavily on lazy evaluation; otherwise, thè evalua- 
tion of (merge-lists flbos (cdr flbos) +) would never terminate: thè input lists are 
effectively infinite. 


Exercise 1 1.3. Define thè sequence of facto rials as an infinite list using delayed 
evaluation. 

Exercise 1 1.4. Describe thè infinite list defined by each of thè following de fini - 
tions. (Check your answers by evaluating thè expressions in LazyCharme.) 

a. (define p (cons 1 (merge-lists p p +))) 

b. (define t (cons 1 (merge-lists t (merge-lists 1 1 +) +))) 

c. (define twos (cons 2 twos)) 

d. (define doubles (merge-lists (ints-from 1) twos *)) 

Exercise 11.5. [**] A simple procedure known as thè Siene of Eratosthenes for 
finding prime numbers was created by Eratosthenes, an ancient Greek math- 
ematician and astronomer. The procedure imagines starting with an (infinite) 
list of all thè integers starting from 2. Then, it repeats thè following two steps 
forever: 

1. Circle thè first number that is not crossed off; it is prime. 

2. Cross off all numbers that are multiples of thè circled number. 

To carry out thè procedure in practice, of course, thè initial list of numbers must 
be finite, otherwise it would take forever to cross off all thè multiples of 2. But, 
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with delayed evaluation, we can implement thè Sieve procedure on an effec- 
tively infinite list. 

Implement thè sieve procedure using lists with lazy evaluation. You may find thè 
list-fllter and merge-lists procedures useful, but will probably find it necessary 
to define some additional procedures. 


11.5 Summary 

Languages are tools for thinking, as well as means to express executable pro- 
grams. A programming language is defined by its grammar and evaluation rules. 
To implement a language, we need to implement a parser that carries out thè 
grammar rules and an evaluator that implements thè evaluation rules. 

We can produce new languages by changing thè evaluation rules of an inter- 
preter. Changing thè evaluation rules changes what programs mean, and en- 
ables new approaches to solving problems. 
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However unapproachable these problems may seem to us and however lielpless we 
stand before them, we bave, nevertheless, thè finn conviction that their solution 
mustfollow by a finite number ofpurely logicai processes. . . This conviction ofthe 
solvability ofevery mathematical problem is a powerful incentive to thè worker. We 
bear within us thè perpetuai cali: There is thè problem. Seek its solution. You can 
find it by pure reason; for in ma thematics there is no ignorabimus. 

David Hilbert, 1900 

In this chapter we consider thè question of what problems can and cannot be 
solved by mechanical computation. This is thè question of computability. a computability 
problem is computable if it can be solved by some algorithm; a problem that is 
noncomputable cannot be solved by any algorithm. 

Section 12.1 considers first thè analogous question for declarative knowledge: 
are there true statements that cannot be proven by any proof? Section 12.2 in- 
troduces thè Halting Problem, a problem that cannot be solved by any algo- 
rithm. Section 12.3 sketches Alan Turing’s proof that thè Halting Problem is 
noncomputable. Section 12.4 discusses how to show other problems are non- 
computable. 

12.1 Mechanizing Reasoning 

Humans have been attempting to mechanize reasoning for thousands of years. 

Aristotle’s Organon developed rules of inference known as syllogisms to codify syllogisms 
logicai deductions in approximately 350 BC. 

Euclid went beyond Aristotle by developing a formai axiomatic System. An ax- 

iomatic System is a formai System consisting of a set of axioms and a set of in- axiomatic system 

ference rules. The goal of an axiomatic System is to codify knowledge in some 

domain. 

The axiomatic System Euclid developed in The Elements concerned construc- 
tions that could be drawn using just a straightedge and a compass. 

Euclid started with five axioms (more commonly known as postulates ); an ex- 
ample axiom is: A straight line segment can be drawn joining any two points. In 
addition to thè postulates, Euclid States five common notions, which could be 
considered inference rules. An example of a common notion is: The whole is 
greater than thè part. 

Starting from thè axioms and common notions, along with a set of definitions 
(e.g., defining a circle), Euclid proved 468 propositions mostly about geometry 
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proposition and number theory. A proposition is a statement that is stated precisely enough 
to be either true or false. Euclid’s fìrst proposition is: given any line, an equilat- 
eral triangle can be constructed whose edges are thè length of that line. 

proof A proof of a proposition in an axiomatic System is a sequence of steps that ends 
with thè proposition. Each step must follow from thè axioms using thè infer- 
ence rules. Most of Euclid’s proofs are constructive: propositions state that a 
thing with a particular property exists, and proofs show steps for constructing 
something with thè stated property. The steps start from thè postulates and fol- 
low thè inference rules to prove that thè constructed thing resulting at thè end 
satisfìes thè requirements of thè proposition. 

consisterà A consisterli axiomatic System is one that can never derive contradictory state - 
ments by starting from thè axioms and following thè inference rules. If a System 
can generate both A and not A for any proposition A, thè System is inconsis- 
tent. If thè System cannot generate any contradictory pairs of statements it is 
consistent. 

complete A complete axiomatic System can derive all true statements by starting from thè 
axioms and following thè inference rules. This means if a given proposition is 
true, some proof for that proposition can be found in thè System. Since we do 
not have a clear definition of true (if we defìned true as something that can be 
derived in thè System, all axiomatic Systems would automatically be complete 
by definition), we state this more clearly by saying that thè System can decide 
any proposition. This means, for any proposition P, a complete axiomatic Sys- 
tem would be able to derive either P or not P. A System that cannot decide all 
statements in thè System is incomplete. An ideal axiomatic System would be 
complete and consistent: it would derive all true statements and no false state- 
ments. 

The completeness of a System depends on thè set of possible propositions. Eu- 
clid’s System is consistent but not complete for thè set of propositions about ge- 
ometry. There are statements that concern simple properties in geometry (a fa- 
mous example is any angle can be divided into three equal sub-angles) that can- 
not be derived in thè System; trisecting an angle requires more powerful tools 
than thè straightedge and compass provided by Euclid’s postulates. 

Figure 12.1 depicts two axiomatic systems. The one on thè left one incomplete'. 
there are some propositions that can be stated in thè System that are true for 
which no valid proof exists in thè System. The one on thè right is inconsistent : 
it is possible to construct valid proofs of both P and notP starting from thè ax- 
ioms and following thè inference rules. Once a single contradictory proposi- 
tion can be proven thè System becomes completely useless. The contradictory 
propositions amount to a proof that true — false, so once a single pair of con- 
tradictory propositions can be proven every other false proposition can also be 
proven in thè System. Hence, only consistent systems are interesting and we 
focus on whether it is possible for them to also be complete. 

Russell’s Paradox. Towards thè end of thè 19 f/i century, many mathematicians 
sought to systematize mathematics by developing a consistent axiomatic Sys- 
tem that is complete for some area of mathematics. One notable attempt was 
Gottlob Frege’s Grundgestze der Arithmetik (1893) which attempted to develop 
an axiomatic System for all of mathematics built from simple logie. 
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Incomplete System: at least one trae 
proposition cannot be proven. 


Inconsistent System: there is a valid 
proof of a proposition that is false. 


Figure 12.1. Incomplete and inconsistent axiomatic Systems. 

Bertrand Russell discovered a problem with Frege’s System, which is now known 

as Russell’s paradox. Suppose R is defined as thè set containing all sets that do Russell’s paradox 

not contain themselves as members. For example, thè set of all prime numbers 

does not contain itself as a member, so it is a member of R. On thè other hand, 

thè set of all entities that are not prime numbers is a member of R. This set 

contains all sets, since a set is not a prime number, so it must contain itself. 

The paradoxical question is: is thè set R a member ofR ? There are two possible 
answers to consider but neither makes sense: 

Yes: R is a member of R 

We defined thè set R as thè set of all sets that do not contain themselves as 
member. Hence, R cannot be a member of itself, and thè statement that R 
is a member of R must be false. 

No: R is not a member of R 

If R is not a member of R, then R does not contain itself and, by defmition, 
must be a member of set R. This is a contradiction, so thè statement that R 
is not a member of R must be false. 

The question is a perfectly clear and precise binary question, but neither thè 
“yes” nor thè “no” answer makes any sense. Symbolically, we summarize thè 
paradox: for any set s, s e R if and only if s £ s. Selecting s — R leads to thè 
contradiction: R e Rii and only if R £ R. 

Whitehead and Russell attempted to resolve this paradox by constructing their 
System to make it impossible to defìne thè set R. Their solution was to introduce 
types. Each set has an associated type, and a set cannot contain members of its 
own type. The set types are defined recursively: 

• A type zero set is a set that contains only non-set objects. 

• A type-n set can only contain sets of type n — 1 and below. 

This defmition avoids thè paradox: thè defmition of R must now define R as a 
set of type k set containing all sets of type k — 1 and below that do not contain 
themselves as members. Since R is a type k set, it cannot contain itself, since it 
cannot contain any type k sets. 
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In 1913, Whitehead and Russell published Principia Mathematica, a bold at- 
tempt to mechanize mathematical reasoning that stretched to over 2000 pages. 
Whitehead and Russell attempted to derive all true mathematical statements 
about numbers and sets starting from a set of axioms and formai inference rules. 
They employed thè type restriction to eliminate thè particular paradox caused 
by set inclusion, but it does not eliminate all self-referential paradoxes. 

For example, consider this paradox named for thè Cretan philosopher Epimenides 
who was purported to have said “All Cretans are liars’’. If thè statement is true, 
than Epimenides, a Cretan, is not a liar and thè statement that all Cretans are 
liars is false. Another version is thè self-referential sentence: this statement is 
false. If thè statement is true, then it is true that thè statement is false (a contra- 
diction). If thè statement is false, then it is a true statement (also a contradic- 
tion) . It was not clear until Godei, however, if such statements could be stated 
in thè Principia Mathematica System. 

12.1.1 Gòdel’s Incompleteness Theorem 

Kurt Godei was born in Brno (then in Austria-Hungary, now in thè Czech Re- 
public) in 1906. Godei proved that thè axiomatic System in Principia Mathemat- 
ica could not be complete and consistent. More generally, Godei showed that 
no powerful axiomatic System could be both complete and consistent: no mat- 
ter what thè axiomatic System is, if it is powerful enough to express a notion of 
proof, it must also be thè case that there exist statements that can be expressed 
in thè System but cannot be proven either true or false within thè System. 

Gòdel’s proof used construction: to prove that Principia Mathematica contains 
statements which cannot be proven either true or false, it is enough to find one 
such statement. The statement Godei found: 

Gpm ■ Statement Gpm does not have any proof in thè System 
of Principia Mathematica. 

Similarly to Russel’s Paradox, this statement leads to a contradiction. It makes 
no sense for Gpm to be either true or false: 

Statement Gpm is provable in thè System. 

If Gpm is proven, then it means Gpm does have a proof, but Gpm stated that 
Gpm has no proof. The System is inconsistent: it can be used to prove a 
statement that is not true. 

Statement Gpm is not provable in thè System. 

Since Gpm cannot be proven in thè System, Gpm is a true statement. The 
System is incomplete: we have a true statement that is not provable in thè 
System. 

The proof generalizes to any axiomatic System, powerful enough to express a 
corresponding statement G: 

G: Statement G does not have any proof in thè System. 

For thè proof to be valid, it is necessary to show that statement G can be ex- 
pressed in thè System. 

To express G formally, we need to consider what it means for a statement to not 
have any proof in thè System. A proof of thè statement G is a sequence of steps, 
T 0 , Ti, T 2 , . . ., T n . Each step is thè set of all statements that have been proven 
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true so far. Initially, To is thè set of axioms in thè System. To be a proof of G, 
T n must contain G. To be a valid proof, each step should be producible from 
thè previous step by applying one of thè inference rules to statements from thè 
previous step. 

To express statement G an axiomatic System needs to be powerful enough to 
express thè notion that a valid proof does not exist. Godei showed that such a 
statement could be constructed using thè Principia Mathematica System, and 
using any System powerful enough to be able to express interesting properties. 
That is, in order for an axiomatic System to be complete and consistent, it must 
be so weak that it is not possible to express this statement has no proof in thè 
System. 

1 2.2 The Halting Problem 

Godei established that no interesting and consistent axiomatic System is capa- 
tile of proving all true statements in thè System. Now we consider thè analogous 
question for computing: are there problemsfor which no algorithm exists? 

Recali these defìnitions form Chapters 1 and 4: 

problem: A description of an input and a desired output. 

procedure: A specification of a series of actions. 

algorithm: A procedure that is guaranteed to always terminate. 

A procedure solves a problem if that procedure produces a correct output for 
every possible input. If that procedure always terminates, it is an algorithm. So, 
thè question can be stated as: are there problems for which no procedure exists 
that produces thè correct output for every possible problem instance in a finite 
amount oftime? 

A problem is computable if there exists an algorithm that solves thè problem. 
It is important to remember that in order for an algorithm to be a solution for 
a problem P, it must always terminate (otherwise it is not an algorithm) and 
must always produce thè correct output for all possible inputs to P. If no such 
algorithm exists, thè problem is noncomputable. 1 

Alan Turing proved that noncomputable problems exist. The way to show that 
uncomputable problems exist is to hnd one, similarly to thè way Godei showed 
unprovable true statements exist by finding an unprovable true statement. 

The problem Turing found is known as thè Halting Problem: 2 

Halting Problem 

Input: A string representing a Python program. 

Output: If evaluating thè input program would ever finish, output True. 
Otherwise, output False. 


lr The terms decidable and undecidable are sometimes used to mean thè same things as com- 
putable and noncomputable. 

2 This problem is a variation on Turing’s originai problem, which assumed a procedure that takes 
one input. Of course, Turing did not deflne thè problem using a Python program since Python had 
not yet been invented when Turing proved thè Halting Problem was noncomputable in 1 936. In fact, 
nothing resembling a programmatile digitai computer would emerge until several years later. 
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Suppose we had a procedure halts that solves thè Halting Problem. The input to 
halts is a Python program expressed as a string. 

For example, halts{\+ 2 3)') should evaluate to True, halts{'wb\\e True: pass') should 
evaluate to False (thè Python pass statement does nothing, but is needed to 
make thè while loop syntactically correct), and 

haltsC"' 

def fibo(n): 

if n == 1 or n == 2: return 1 
else: return flbo{n—l) + flbo[n—2) 
flbo{60) 

I II ! I I ^ 

should evaluate to True. From thè last example, it is clear that halts cannot be 
implemented by evaluating thè expression and outputting True if it terminates. 
The problem is knowing when to give up and output False. As we analyzed in 
Chapter 7, evaluating fibo{ 60) would take trillions of years; in theory, though, it 
eventually fmishes so halts should output True. 

This argument is not suffìcient to prove that halts is noncomputable. It just 
shows that one particular way of implementing halts would not work. To show 
that halts is noncomputable, we need to show that it is impossible to implement 
a halts procedure that would produce thè correct output for all inputs in a finite 
amount of time. 

Here is another example that suggests (but does not prove) thè impossibility of 
halts (where sumOJTwoPrimes is defined as an algorithm that take a number as 
input and outputs True if thè number is thè sum of two prime numbers and False 
otherwise): 

halts['r\ = 4; while sumOfTwoPrimes(n): n = n + 2’) 

This program halts if there exists an even number greater than 2 that is not thè 
sum of two primes. We assume unbounded integers even though every actual 
computer has a limit on thè largest number it can represent. Our computing 
model, though, uses an infinite tape, so there is no arbitrary limit on number 
sizes. 

Knowing whether or not thè program halts would settle an open problem known 
as Goldbach’s Conjecture: every even integer greater than 2 can be written as thè 
sum oftwo primes. Christian Goldbach proposed a form of thè conjecture in a 
letter to Leonhard Euler in 1742. Euler refined it and believed it to be true, but 
couldn’t prove it. 

With a halts algorithm, we could settle thè conjecture using thè expression above: 
if thè result is False, thè conjecture is proven; if thè result is True, thè conjecture 
is disproved. We could use a halts algorithm like this to resolve many other open 
problems. This strongly suggests there is no halts algorithm, but does not prove 
it cannot exist. 

Proving Noncomputability. Proving non-existence is requires more than just 
showing a hard problem could be solved if something exists. One way to prove 
non-existence of an X, is to show that if an X exists it leads to a contradiction. 
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We prove that thè existence of a halts algorithm leads to a contradiction, so no 
halts algorithm exists. 

We obtain thè contradiction by showing one input for which thè halts procedure 
could not possibly work correctly. Consider this procedure: 

def paradoxQ : 

if haltsC paradox()'): while True: pass 


The body of thè paradox procedure is an if expression. The consequent expres- 
sion is a never-ending loop. 

The predicate expression cannot sensibly evaluate to either True or False: 
haltsC paradox()’) =>- True 

If thè predicate expression evaluates to True, thè consequent block is eval- 
uated producing a never-ending loop. Thus, if haltsC paradoxQ') evaluates 
to True, thè evaluation of an application of paradox never halts. But, this 
means thè result of haZtó('paradox()') was incorrect. 

haltsC paradox()’) => False 

If thè predicate expression evaluates to False, thè alternate block is evalu- 
ated. It is empty, so evaluation terminates. Thus, thè evaluation of paradox () 
terminates, contradicting thè result of haZfó('paradox()'). 

Either result for haltsC paradox ()') leads to a contradiction! The only sensible 
thing halts could do for this input is to not produce a value. That means there 
is no algorithm that solves thè Halting Problem. Any procedure we define to 
implement halts must sometimes either produce thè wrong result or fail to pro- 
duce a result at all (that is, run forever without producing a result) . This means 
thè Halting Problem is noncomputable. 

There is one important hole in our proof: we argued that because paradox does 
not make sense, something in thè definition of paradox must not exist and iden- 
tified halts as thè component that does not exist. This assumes that everything 
else we used to define paradox does exist. 

This seems reasonable enough — they are built-in to Python so they seem to ex- 
ist. But, perhaps thè reason paradox leads to a contradiction is because True 
does not really exist or because it is not possible to implement an if expression 
that strictly follows thè Python evaluation rules. Although we have been using 
these and they seems to always work fine, we have no formai model in which 
to argue that evaluating True always terminates or that an if expression means 
exactly what thè evaluation rules say it does. 

Our informai proof is also insufficient to prove thè stronger claim that no algo- 
rithm exists to solve thè halting problem. All we have shown is that no Python 
procedure exists that solves halts. Perhaps there is a procedure in some more 
powerful programming language in which it is possible to implement a solution 
to thè Halting Problem. In fact, we will see that no more powerful programming 
language exists. 

A convincing proof requires a formai model of computing. This is why Alan Tur- 
ing developed a model of computation. 


244 


12.3. Universality 


12.3 Universality 

Recali thè Turing Machine model from Chapter 6: a Turing Machine consists 
of an infinite tape divided into discrete square into which symbols from a hxed 
alphabet can be written, and a tape head that moves along thè tape. On each 
step, thè tape head can read thè Symbol in thè current square, write a Symbol in 
thè current square, and move left or right one square or halt. The machine can 
keep track of a finite number of possible States, and determines which action to 
take based on a set of transition rules that specify thè output Symbol and head 
action for a given current state and read Symbol. 

Turing argued that this simple model corresponds to our intuition about what 
can be done using mechanical computation. Recali this was 1936, so thè model 
for mechanical computation was not what a mechanical computer can do, but 
what a human computer can do. Turing argued that his model corresponded 
to what a human computer could do by following a systematic procedure: thè 
infinite tape was as powerful as a two- dimensionai sheet of paper or any other 
recording medium, thè set of symbols must be finite otherwise it would not be 
possible to correctly distinguish all symbols, and thè number of machine States 
must be finite because there is a limited amount a human can keep in mind at 
one time. 

We can enumerate all possible Turing Machines. One way to see this is to de- 
vise a notation for writing down any Turing Machine. A Turing Machine is com- 
pletely described by its alphabet, States and transition rules. We could write 
down any Turing Machine by numbering each state and listing each transition 
rule as a tuple of thè current state, alphabet Symbol, next state, output Symbol, 
and tape direction. We can map each state and alphabet Symbol to a number, 
and use this encoding to write down a unique number for every possible Turing 
Machine. Hence, we can enumerate all possible Turing Machines by just enu- 
merating thè positive integers. Most positive integers do not correspond to valid 
Turing Machines, but if we go through all thè numbers we will eventually reach 
every possible Turing Machine. 

This is step towards proving that some problems cannot be solved by any algo- 
rithm. The number of Turing Machines is less than thè number of reai numbers. 
Both numbers are infinite, but as explained in Section 1.2.2, Cantor’s diagonal- 
ization proof showed that thè reai numbers are not countable. Any attempt to 
map thè reai numbers to thè integers must fail to include all thè reai numbers. 
This means there are reai numbers that cannot be produced by any Turing Ma- 
chine: there are fewer Turing Machines than there are reai numbers, so there 
must be some reai numbers that cannot be produced by any Turing Machine. 

The next step is to define thè machine depicted in Figure 12.2. A Universal Tur- 
Universal Turing ing Machine is a machine that takes as input a number that identifies a Turing 
Machine Machine and simulates thè specified Turing Machine running on initially empty 
input tape. 

The Universal Turing Machine can simulate any Turing Machine. In his proof, 
Turing describes thè transition rules for such a machine. It simulates thè Turing 
Machine encoded by thè input number. One can imagine doing this by using 
thè tape to keep track of thè state of thè simulated machine. For each step, thè 
universal machine searches thè description of thè input machine to find thè ap- 
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Figure 12.2. Universal Turing Machine. 


propriate rule. This is thè rule for thè current state of thè simulateci machine 
on thè current input Symbol of thè simulateci machine. The universal machine 
keeps track of thè machine and tape state of thè simulated machine, and simu- 
lates each step. Thus, there is a single Turing Machine that can simulate every 
Turing Machine. 

Since a Universal Turing Machine can simulate every Turing Machine, and a Tur- 
ing Machine can perform any computation according to our intuitive notion of 
computation, this means a Universal Turing Machine can perform all computa- 
tions. Using thè universal machine and a diagonalization argument similar to 
thè one above for thè reai numbers, Turing reached a similar contradiction for a 
problem analogous to thè Halting Problem for Python programs but for Turing 
Machines instead. 

universal 
programming 
language 


12.4 Proving Non- Computability 

We can show that a problem is computable by describing a procedure and prov- 
ing that thè procedure always terminates and always produces thè correct an- 
swer. It is enough to provide a convincing argument that such a procedure ex- 
ists; finding thè actual procedure is not necessary (but often helps to make thè 
argument more convincing) . 

To show that a problem is not computable, we need to show that no algorithm 
exists that solves thè problem. Since there are an infinite number of possible 
procedures, we cannot just list all possible procedures and show why each one 
does not solve thè problem. Instead, we need to construct an argument showing 
that if there were such an algorithm it would lead to a contradiction. 

The core of our argument is based on knowing thè Halting Problem is noncom- 
putable. If a solution to some new problem P could be used to solve thè Halting 


If we can simulate a Universal Turing Machine in a programming language, that 
language is a universal programming language. There is some program that can 
be written in that language to perform every possible computation. 

To show that a programming language is universal, it is sufficient to show that 
it can simulate any Turing Machine, since a Turing Machine can perform ev- 
ery possible computation. To simulate a Universal Turing Machine, we need 
some way to keep track of thè state of thè tape (for example, thè list datatypes 
in Scheme or Python would be adequate), a way to keep track of thè internai 
machine state (a number can do this), and a way to execute thè transition rules 
(we could define a procedure that does this using an if expression to make de- 
cisions about which transition rule to follow for each step), and a way to keep 
going (we can do this in Scheme with recursive applications) . Thus, Scheme is a 
universal programming language: one can write a Scheme program to simulate 
a Universal Turing Machine, and thus, perform any mechanical computation. 
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Problem, then we know that P is also noncomputable. That is, no algorithm 
exists that can solve P since if such an algorithm exists it could be used to also 
solve thè Halting Problem which we already know is impossible. 

Reduction Proofs. The proof technique where we show that a solution for some 
reduction problem P can be used to solve a different problem Q is known as a reduction. 

reducible A problem Q is reducible to a problem P if a solution to P could be used to solve 
Q. This means that problem Q is no harder than problem P, since a solution to 
problem Q leads to a solution to problem P. 


Example 12.1: Prints-Three Problem 


Consider thè problem of determining if an application of a procedure would 
ever print 3: 

Prints-Three 

Input: A string representing a Python program. 

Output: If evaluating thè input program would print 3, output True; oth- 
erwise, output False. 

We show thè Prints-Three Problem is noncomputable by showing that it is as 
hard as thè Halting Problem, which we already know is noncomputable. 

Suppose we had an algorithm printsThree that solves thè Prints-Three Problem. 
Then, we could defrne halts as: 

def haltsip ): 

return printsThreelp + print(3)') 

The printsThree application would evaluate to True if evaluating thè Python pro- 
gram specified by p would halt since that means thè print(3) statement ap- 
pended to p would be evaluated. On thè other hand, if evaluating p would not 
halt, thè added print statement never evaluated. As long as thè program speci- 
fied by p would never print 3, thè application of printsThree should evaluate to 
False. Hence, if a printsThree algorithm exists, we would use it to implement an 
algorithm that solves thè Halting Problem. 

The one wrinkle is that thè specified input program might print 3 itself. We can 
avoid this problem by transforming thè input program so it would never print 
3 itself, without otherwise altering its behavior. One way to do this would be to 
replace all occurrences of print (or any other built-in procedure that prints) in 
thè string with a new procedure, dontprint that behaves like print but doesn’t 
actually print out anything. Suppose thè replacePrints procedure is defined to 
do this. Then, we could use printsThree to define halts-. 

def halis(p): return prinlsThree(replacePrinls(p) + print(3)’) 

We know that thè Halting Problem is noncomputable, so this means thè Prints- 
Three Problem must also be noncomputable. 


Exploration 12.1: Virus Detection 


The Halting Problem and Prints-Three Problem are noncomputable, but do seem 
to be obviously important problems. It is useful to know if a procedure appli- 
cation will terminate in a reasonable amount of time, but thè Halting Problem 
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does not answer that question. It concerns thè question of whether thè proce- 
dure application will terminate in any finite amount of time, no matter how long 
it is. This example considers a problem for which it would be very useful to have 
a solution for it one existed. 

A virus is a program that infects other programs. A virus spreads by copying its 
own code into thè code of other programs, so when those programs are executed 
thè virus will execute. In this manner, thè virus spreads to infect more and more 
programs. A typical virus also includes a malicious payload so when it executes 
in addition to infecting other programs it also performs some damaging (cor- 
rupting data files) or annoying (popping up messages) behavior. The Is-Virus 
Problem is to determine if a procedure specification contains a virus: 

Is-Virus 

Input: A specification of a Python program. 

Output: If thè expression contains a virus (a code fragment that will infect 
other files) output True. Otherwise, output False. 

We demonstrate thè Is-Virus Problem is noncomputable using a similar strat- 
egy to thè one we used for thè Prints-Three Problem: we show how to define a 
halts algorithm given a hypothetical isVirus algorithm. Since we know halts is 
noncomputable, this shows there is no isVirus algorithm. 

Assume infectFiles is a procedure that infects files, so thè result of evaluating 
zWzrws('infectFiles()') is True. We could define halts as: 

def halts(p): 

return isVirusip + infectFiles()') 

This works as long as thè program specified by p does not exhibit thè file-infecting 
behavior. If it does, p could infect a file and never terminate, and halts would 
produce thè wrong output. To solve this we need to do something like we did in 
thè previous example to hide thè printing behavior of thè originai program. 

A rough definition of file-infecting behavior would be to consider any write to 
an executable file to be an infection. To avoid any file infections in thè spe- 
cific program, we replace all procedures that write to files with procedures that 
write to shadow copies of these files. For example, we could do this by creating 
a new temporary directory and prepend that path to all file names. We cali this 
(assumed) procedure, sandBox, since it transforms thè originai program speci- 
fication into one that would execute in a protected sandbox. 

def haltsip ): is Vi rusisa / ulBoxi p) + infectFiles()') 

Since we know there is no algorithm that solves thè Halting Problem, this proves 
that there is no algorithm that solves thè Is-Virus problem. 

Virus scanners such as Symantec’s Norton AntiVirus attempt to solve thè Is- 
Virus Problem, but its non-computability means they are doomed to always fail. 
Virus scanners detect known viruses by scanning files for strings that match sig- 
natures in a database of known viruses. As long as thè signature database is 
frequently updated they may be able to detect currently spreading viruses, but 
this approach cannot detect a new virus that will not match thè signature of a 
previously known virus. 
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Sophisticated virus scanners employ more advanced techniques to attempt to 
detect complex viruses such as metamorphic viruses that alter their own code 
as they propagate to avoid detection. But, because thè generai Is-Virus Prob- 
lem is noncomputable, we know that it is impossible to create a program that 
always terminates and that always correctly determines if an input procedure 
specifìcation is a virus. 


I am rather puzzled 
whyyou draw this 
distinction between 
prooffinders and 
proof checkers. It 
seems to me rather 
unimportant as one 
can always get a 
proof flnder from a 
proof checker, and 
thè converse is 
almost true: thè 
converse false iffor 
instance one allows 
thè proof flnder to 
go through a proof 
in thè ordinaiy way, 
and then, rejecting 
thè steps, to write 
down thè final 
formula as a 'proof' 
ofitself. One can 
easily think up 
sidtable restrictions 
on thè idea of proof 
which udii make 
this converse true 
and which agree 
well with our ideas 
ofwhat a proof 
should be like. I am 
afraid this may be 
more confusing to 
you than 
enlightening. 
Alan Turing, letter to 
MaxNewman, 1940 


Exercise 12.1. Is thè Launches-Missiles Problem described below computable? 
Provide a convincing argument supporting your answer. 

Launches-Missiles 

Input: A specifìcation of a procedure. 

Output: If an application of thè procedure would lead to thè missiles be- 
ing launched, outputs True. Otherwise, outputs False. 

You may assume that thè only thing that causes thè missiles to be launched is 
an application of thè launchMissiles procedure. 

Exercise 12.2. Is thè Same-Result Problem described below computable? Pro- 
vide a convincing argument supporting your answer. 

Same-Result 

Input: Specifications of two procedures, P and Q. 

Output: If an application of P terminates and produces thè same value 
as applying Q, outputs True. If an application of P does not terminate, 
and an application of Q also does not terminate, outputs True. Otherwise, 
outputs False. 

Exercise 12.3. Is thè Check- Proof Problem described below computable? Pro- 
vide a convincing argument supporting your answer. 

Check-Proof 

Input: A specifìcation of an axiomatic System, a statement (thè theorem), 
and a proof (a sequence of steps, each identifying thè axiom that is ap- 
plied) . 

Output: Outputs True if thè proof is a valid proof of thè theorem in thè 
System, or False if it is not a valid proof. 


Exercise 12.4. Is thè Find-Finite-Proof Problem described below computable? 
Provide a convincing argument supporting your answer. 

Find-Finite-Proof 

Input: A specifìcation of an axiomatic System, a statement (thè theorem), 
and a maximum number of steps (max-steps). 

Output: If there is a proof in thè axiomatic System of thè theorem that 
uses max-steps or fewer steps, outputs True. Otherwise, outputs False. 
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Exercise 12.5. [*] Is thè Find-Proof Problem described below computable? Pro- 
vide a convincing argument why it is or why it is not computable. 

Find-Proof 

Input: A specibcation of an axiomatic System, and a statement (thè theo- 
rem). 

Output: If there is a proof in thè axiomatic System of thè theorem, outputs 
True. Otherwise, outputs False. 


Exploration 12.2: BusyBeavers 


Considerthe Busy-Beaver Problem (devised by Tibor Rado in 1962): 

Busy-Beaver 

Input: A positive integer, n. 

Output: A number representing that maximum number of steps a Turing 
Machine with n States and a two-symbol tape alphabet can run starting 
on an empty tape before halting. 

We use 0 and 1 for thè two tape symbols, where thè blank squares on thè tape are 
interpreted as Os (alternately, we could use blank and X as thè symbols, but it is 
more naturai to describe machines where symbols are 0 and 1, so we can think 
of thè initially blank tape as containing all Os). 

For example, if thè Busy Beaver input n is 1, thè output should be 1. The best 
we can do with only one state is to halt on thè first step. If thè transition rule 
for a 0 input moves left, then it will reach another 0 square and continue forever 
without halting; similarly it if moves right. 

For n — 2, there are more options to consider. The machine in Figure 12.3 runs 
for 6 steps before halting, and there is no two-state machine that runs for more 
steps. One way to support this claim would be to try simulating all possible two- 
state Turing Machines. 



1/1/L 


l/l,Halt 
► 



Figure 12.3. Two-state Busy Beaver Machine. 


Busy Beaver numbers increase extremely quickly. The maximum number of 
steps for a three-state machine is 21, and for a four-state machine is 107. The 
value for a fìve-state machine is not yet known, but thè best machine found to 
date runs for 47,176,870 steps! For six States, thè best known result, discovered 
in 2007 by Terry Ligocki and Shawn Ligocki, is over 2879 decimai digits long. 
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We can prove thè Busy Beaver Problem is noncomputable by reducing thè Halt- 
ing Problem to it. Suppose we had an algorithm, bh( ri ) , that takes thè number 
of States as input and outputs thè corresponding Busy Beaver. Then, we could 
solve thè Halting Problem for a Turing Machine: 


TM Halting Problem 
Input: A string representing a Turing Machine. 


Output: If executing thè input Turing Machine starting with a blank tape 
would ever finish, output True. Otherwise, output False. 


The TM Halting Problem is different from thè Halting Problem as we defìned 
it earlier, so fìrst we need to show that thè TM Halting Problem is noncom- 
putable by showing it could be used to solve thè Python Halting Problem. Be- 
cause Python is universal programming language, it is possible to transform any 
Turing Machine into a Python program. Once way to do this would be to write 
a Universal Turing Machine simulator in Python, and then create a Python pro- 
gram that fìrst creates a tape containing thè input Turing Machine description, 
and then calls thè Universal Turing Machine simulator on that input. This shows 
that thè TM Halting Problem is noncomputable. 

Next, we show that an algorithm that solves thè Busy Beaver Problem could be 
used to solve thè TM Halting Problem. Here’s how (in Pythonish pseudocode): 


















def /: 

tal 

tsTM(m): 


States = numberOfStat es ( m ) 
maxSteps = bb(s lai.es) 
state = 0 
tape - [] 

for step in range{ 0, maxSteps ) : 
state, tape - s i m alale O n eS tep[m, state, tape) 
if halted(state): return True 
return False 


The simulateOneStep procedure takes as inputs a Turing Machine description, 
its current state and tape, and simulates thè next step on thè machine. So, 
haltsTM simulates up to bb(n) steps of thè input machine m where n is thè num- 
ber of States in m. Since bb(n) is thè maximum number of steps a Turing Ma- 
chine with n States can execute before halting, we know if m has not halted in 
thè simulate before maxSteps is reached that thè machine m will never halt, and 
can correctly return False. This means there is no algorithm that can solve thè 
Busy Beaver Problem. 


Exercise 12.6. Confimi that thè machine showing in Figure 12.3 runs for 6 steps 
before halting. 
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Exercise 12.7. Prove thè Beaver Bound problem described below is also non- 
computable: 

Beaver-Bound 

Input: A positive integer, n. 

Output: A number that is greater than thè maximum number of steps a 
Turing Machine with n States and a two-symbol tape alphabet can run 
starting on an empty tape before halting. 

A valid solution to thè Beaver-Bound problem can produce any result for n as 
long as it is greater than thè Busy Beaver value for n. 

Exercise 12.8. [***] Find a 5-state Turing Machine that runs for more than 
47,176,870 steps, or prove that no such machine exists. 


12.5 Summary 

Although today’s computers can do amazing things, many of which could not 
even have been imagined twenty years ago, there are problems that can never 
be solved by computing. The Halting Problem is thè most famous example: 
it is impossible to defìne a mechanical procedure that always terminates and 
correctly determines if thè computation specihed by its input would terminate. 
Once we know thè Halting Problem is noncomputable, we can show that other 
problems are also noncomputable by illustrating how a solution to thè other 
problem could be used to solve thè Halting Problem which we know to be im- 
possible. 

Noncomputable problems frequently arise in practice. For example, identify- 
ing viruses, analyzing program paths, and constructing proofs, are all noncom- 
putable problems. 

Just because a problem is noncomputable does not mean we cannot produce 
useful programs that address thè problem. These programs provide approxi- 
mate Solutions, which are often useful in practice. They produce thè correct re- 
sults on many inputs, but on some inputs must either fail to produce any result 
or produce an incorrect result. 
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